Embedded, Programming, Ubuntu

No comments

These are instructions on how to set up a usable development environment on Ubuntu Linux 13.10 64 bits, complete with GDB debugging over J-Link, for the EFM32 series development boards by Energy Micro.

Start by download the Simplicity Studio beta installer for Linux:

mkdir -p ~/simplicity
cd ~/simplicity
wget http://cdn.energymicro.com/dl/packages/studio_linux.tar.gz
tar -xvf studio_linux.tar.gz
cd energymicro
python studio.py

The script will do its process and eventually it will error out saying that “Simplicity Studio New” was not found. This is ok, the process can continue anyway:

cd studio_new
chmod +x studio.py
./studio.py

Now proceed to install everything you are offered (it’ll take some time).

The next thing you need is the J-Link software to be able to connect to the board via JTAG over USB. Go to Segger’s website and download the package titled “Software and documentation pack for Linux V4.80h, DEB Installer 32-bit version“. Do note that you have to get the 32 bit version even if you have a 64 bit Linux OS.

sudo dpkg -i jlink_4.80.8_i386.deb

You also need a toolchain to compile the examples (and your own code) to ARM. The toolchain required is called arm-none-eabi (ARM processor, no distro/bare metal, “eabi” ABI specification). There’s an Ubuntu PPA that provides this. There’s no packages for Ubuntu 13.10, but those for 13.04 work just fine, so you can do:

sudo add-apt-repository ppa:terry.guo/gcc-arm-embedded

And then edit /etc/apt/sources.list.d/terry_guo-gcc-arm-embedded-saucy.list replacing “saucy” by “raring”, so the file ends up like this:

deb http://ppa.launchpad.net/terry.guo/gcc-arm-embedded/ubuntu raring main

 

After that:

sudo apt-get update
sudo apt-get install gcc-arm-none-eabi

For convenience, you can set up a few aliases in your ~/.bashrc or ~/.bash_profile file:

alias efm32gdb="/usr/bin/arm-none-eabi-gdb"
alias efm32gdbserver="JLinkGDBServer -if SWD"

The first alias just points the command efm32gdb to the right gdb version and the other alias starts a GDB server over J-Link using the Single Wire protocol.

At this point you should be pretty much set up. If you want to compile Simplicity Studio’s examples, use the armgcc Makefiles provided, setting the LINUXCS environment variable to “/usr”

LINUXCS=/usr Makefile -f Makefile.someproject

 

If the process completes successfully, you’ll get two files in the exe folder. something.bin is the file you should flash using Simplicity Studio’s Energy Commander “Flash” panel. something.out is the file you can use for debugging.

efm32gdbserver &
efm32gdb -x mygdbinitscript

mygdbinitscript should look like:

file armgcc/exe/something.out
target remote :2331
#set tdesc filename target-m3.xml
mon flash download = 1
#mon flash device = EFM32GG990F1024
set mem inaccessible-by-default off
#set remote memory-read-packet-size 1200
#set remote memory-read-packet-size fixed
mon speed 4000
mon endian little
mon reset 1
tbreak main
cont

As a matter of personal taste I use KDevelop as my IDE, because it makes integrating all this in your own project dead simple. All in all it takes some effort to set up the environment, but once you do everything works like a charm, and you can debug seamlessly from within a top notch IDE. I also did try using QtCreator, but given it does not allow you (as of this writing) to customize the GDB binary, you can not do remote debugging as you do with KDevelop.

Javascript, Meteor, Node

No comments

Stradjectives for Public Art League is a Meteor based web app we’ve developed at 2wav . Before starting, we knew that we wanted to create a modern HTML5+CSS3 app, prioritizing a dynamic behavior. Initially we started looking for open source projects that would allow us to create a Nodejs+Mongoose+Backbone+SocketIO solution in the short amount of time we had before the launch. Such alternatives do exist, but unfortunately none that were mature enough to pin our hopes and dreams on them. We did delve into Derbyjs initially, but sadly it proved too much for us to handle.

Then, along came Meteor. It provided everything we wanted, it has clear documentation, and where the documentation is not there, there’s a big community around it to help out. If you are looking to quickly develop a dynamic web app that “just works” and is fully data reactive, my advice is that you do give Meteor a chance.

Of course, when it came down to the implementation, some issues did come up. Wrangling the reactiveness of the app so you don’t get whole page re-renders for a minor change in the database is not trivial and it takes some time to figure out how it all works and what’s the best balance between reactiveness and performance/usability (this is a subject for another post, but let’s just say that you’ll eventually arrive at the conclusion that client side animations and data reactiveness are fundamentally at odds with each other).

Another problem we encountered with Meteor is that it’s very easy to add dependencies and before you notice it, the app gets fairly big, even with minification, gzipping, etc. This, added to the big images the app uses, required that we showed some sort of splash screen quickly upon access to prevent the user from thinking the site wasn’t responsive.

Meteor (at least as of version 0.6.2) has no functionality to do this, and no easy way to hack into its build system. What we ended up doing is a dirty hack, but one that should tide you over if you find yourself in the same spot, at least until an equivalent feature is properly implemented.

What we did is a monkey patch of sorts. Meteor fetches all the app dependencies and makes an “app.html” file that acts as the main file for your app (it loads all the CSS and Javascript). On our server code, we open that file, modify it to embed a base64 encoded image, and insert it “manually” in the document’s body (the document body doesn’t even exist at this point, so we create that too). The image is base64 encoded right into the html to make sure it’ll be shown before all the Javascript files are loaded. Otherwise, depending on the browser, the splash image can potentially be loaded after other files, thus taking longer to show up, defeating the purpose of having something show up right away.

The code is as follows:

// Paste this at the end of server.js
// This creates a base64 encoded version of splash.png
// The reason behind this is to make sure the splash image shown *before* the rest of the CSS+JS starts to load
// Otherwise the request for splash.png gets queued with the rest of the files and it takes a good while to show up

var require = __meteor_bootstrap__.require ? __meteor_bootstrap__.require : Npm.require;
var fs = require('fs');
var path = require('path');

var splash_path = path.join(path.resolve("."), "public/images/splash.png")
var splash_img = fs.readFileSync(splash_path);

var splash_base64 = new Buffer(splash_img).toString('base64');

// Hacks are not over yet, here we patch app.html to include this splash before anything else.

var app_html_path = path.join(path.resolve("."), ".meteor/local/build/app.html")
var app_html = fs.readFileSync(app_html_path, 'utf8');
app_html = app_html.replace("<body>", "");
app_html = app_html.replace("</body>", "");
app_html = app_html.replace(
    "// ##RUNTIME_CONFIG##",
    // Insert the splash image
    "var _body = document.createElement('body');\n" +
    "_body.innerHTML = '<div class=\"page loading\"><div id=\"splash-content\"><img id=\"splash-image\" src=\"data:image/png;base64,"+splash_base64+"\" /><div id=\"splash-loader\"></div></div></div>';\n" +
    "document.documentElement.appendChild(_body);\n" +
    "__meteor_runtime_config__ = " + JSON.stringify(__meteor_runtime_config__) + ";");

fs.writeFileSync(app_html_path, app_html);

There’s one minor side effect to this hack. When all is said and done, you end up with two body elements in your HTML. This didn’t prove to be a problem, we just remove the second one with:

$('body').eq(1).remove();

 

HTML5, Programming

No comments

Every now and then I’m forced to exit the Candyland Paradise known as Python and I have to lower myself into the muddy waters of Javascript…the things I do for money.

Anyway, while developing an HTML5 app for 2wav, which makes heavy use of CSS3 transforms via the Barajas plugin (do check it out, it’s great), I found that trying to drag a card using jQuery UI’s draggable component doesn’t quite work. There’s even a bug report about it, which sadly is marked as “won’t fix”, so you have to resort to workarounds to make it behave as expected.

The workaround looks like this:

    $(li).draggable({ 
                        revert: true,
                        start: function() {
                            /* Temporarily revert the transform so drag and dropping works as expected */
                            var parentRect = $(this).parent()[0].getBoundingClientRect();
                            var rect = this.getBoundingClientRect();
                            /* cssBrowserPrefix is one of: -moz -webkit -o */
                            $(this).css(cssBrowserPrefix + 'transition', 'all 0 ease 0');
                            $(this).css('transform', 'none');
                            $(this).css('left', rect['left']-parentRect['left']);
                          },
                        stop: function() {
                          /* Revert the transformation changes done on start, if needed */
                        }
                    });

What this does is reset the transition (so the change is done immediately) and the transform properties, and replaces the former for the equivalent value in the left property. I’m not doing rotation, scaling or Y axis translation here, solving the problem in those cases may be a little more involved but the principle is the same.

As a side note, in case you are interested in knowing about the underlying technical details, the reason why draggable has problems with CSS transforms is because these transforms operate in a kind of “one way” after all the other CSS properties have been applied to the element (which probably stems from the fact that they are usually implemented using 3D hardware accelerated operations; that is the rotation, scaling and translation is done directly on the GPU). The bottom line is that if you use a translateX transform, the left CSS property will remain unaffected, given the transformation matrix is applied onto the element after it’s been positioned on the page according to all the other HTML and CSS rules.

While this method is way faster than computing and doing the transformation on the CPU,  a common problem with this approach appears when eventually the user (and the browser itself, in case it needs to apply mouse interaction such as hovering) may need to know where the element ended up visually, for example, to drag it around. So, to get this final position you have to call  getBoundingClientRect as illustrated in the code snippet.

 

 

Programming

5 comments

I’ve looked everywhere for a good embeddable HTML+CSS rendering engine to integrate with my game engine Ignifuga, so my faithful users (all 3 of you!) could have the flexibility of developing GUIs (or even the whole game) using familiar concepts from the web development arena.

Webkit is nice, fast, well supported and extensively used for embedding, but it’s licensed under GPL, so it was of no use. Gecko has a more liberal license, but between contradictory reports of Mozilla pulling the plug on embedding support and the outdated documentation (and very complex API!), I felt that integrating it would be more work than I could take on, as neither of these engines are really oriented to the sort of use I had in mind.

Luckily, I found libRocket. It’s not strictly HTML+CSS, but it’s very, very similar, MIT licensed, and while it’s not very actively developed, it’s still mantained as far as I can tell. And it’s Python scriptable to boot! Folding the library into the Ignifuga build system proved to be simple enough (provided a few minor fixes that mostly solved path problems), and integration with the Python machinery is dead easy thanks to Boost (yes, I also had to bring along Boost…kinda sucks, but I’ve made a promise to myself to eventually work out a Cython based binding for libRocket).

Having cleared up those initial hurdles, it came down to integrate the libRocket renderer with SDL2, the user input and rendering backend (among other things) on top of which Ignifuga exists. SDL2 abstracts texture rendering across platforms, and internally it uses DirectX 9.0/OpenGL/GL ES/GL ES2 depending on what’s available on the target platform. libRocket provides a simple mechanism for integrating the renderer, and a few ready to use examples can be found in the source code and in their forums. In a nutshell, what you need to do is provide libRocket with functions to create textures from image files, to render a set of triangles, and to feed it user input.

Doing this proved to be difficult, as SDL can be used to do mostly everything required by libRocket, but it kept important stuff, such as the OpenGL textures, locked in internally (as it’s not designed to be integrated with 3rd party libraries in this particular way). So, with permission from Sam Lantinga I added a couple of useful functions that make this sort of integration possible (I’ve since heard of people trying to integrate Cairo in this same manner). The functions are SDL_GL_BindTexture and SDL_GL_UnbindTexture, which bind/unbind an SDL texture to the current OpenGL context, thus allowing regular OpenGL calls to operate on SDL created textures.

Using these functions, the relevant rendering code looks like this:

void RocketSDLRenderInterfaceOpenGL::RenderGeometry(Rocket::Core::Vertex* vertices, int num_vertices, int* indices, int num_indices, const Rocket::Core::TextureHandle texture, const Rocket::Core::Vector2f& translation)
{
    // SDL uses shaders that we need to disable here
    render_data.glUseProgramObjectARB(0);
    render_data.glPushMatrix();
    render_data.glTranslatef(translation.x, translation.y, 0);

    std::vector Positions(num_vertices);
    std::vector Colors(num_vertices);
    std::vector TexCoords(num_vertices);
    float texw, texh;

    SDL_Texture* sdl_texture = NULL;
    if(texture)
    {
        render_data.glEnableClientState(GL_TEXTURE_COORD_ARRAY);
        sdl_texture = (SDL_Texture *) texture;
        SDL_GL_BindTexture(sdl_texture, &texw, &texh);
    }

    for(int  i = 0; i < num_vertices; i++) {
        Positions[i] = vertices[i].position;
        Colors[i] = vertices[i].colour;
        if (sdl_texture) {
            TexCoords[i].x = vertices[i].tex_coord.x * texw;
            TexCoords[i].y = vertices[i].tex_coord.y * texh;
        }
        else TexCoords[i] = vertices[i].tex_coord;
    };

    render_data.glEnableClientState(GL_VERTEX_ARRAY);
    render_data.glEnableClientState(GL_COLOR_ARRAY);
    render_data.glVertexPointer(2, GL_FLOAT, 0, &Positions[0]);
    render_data.glColorPointer(4, GL_UNSIGNED_BYTE, 0, &Colors[0]);
    render_data.glTexCoordPointer(2, GL_FLOAT, 0, &TexCoords[0]);

    render_data.glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE);
    render_data.glEnable(GL_BLEND);
    render_data.glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
    render_data.glDrawElements(GL_TRIANGLES, num_indices, GL_UNSIGNED_INT, indices);
    render_data.glDisableClientState(GL_VERTEX_ARRAY);
    render_data.glDisableClientState(GL_COLOR_ARRAY);

    if (sdl_texture) {
        SDL_GL_UnbindTexture(sdl_texture);
        render_data.glDisableClientState(GL_TEXTURE_COORD_ARRAY);
    }

    render_data.glColor4f(1.0, 1.0, 1.0, 1.0);
    render_data.glPopMatrix();
    /* Reset blending and draw a fake point just outside the screen to let SDL know that it needs to reset its state in case it wants to render a texture */
    render_data.glDisable(GL_BLEND);
    SDL_SetRenderDrawBlendMode(renderer, SDL_BLENDMODE_NONE);
    SDL_RenderDrawPoint(renderer, -1, -1);
void RocketSDLRenderInterfaceOpenGLES::RenderGeometry(Rocket::Core::Vertex* vertices, int num_vertices, int* indices, int num_indices, const Rocket::Core::TextureHandle texture, const Rocket::Core::Vector2f& translation)
{
    render_data.glPushMatrix();
    render_data.glTranslatef(translation.x, translation.y, 0);

    std::vector Positions(num_vertices);
    std::vector Colors(num_vertices);
    std::vector TexCoords(num_vertices);
    float texw, texh;

    SDL_Texture* sdl_texture = NULL;
    if(texture)
    {
        render_data.glEnableClientState(GL_TEXTURE_COORD_ARRAY);
        sdl_texture = (SDL_Texture *) texture;
        SDL_GL_BindTexture(sdl_texture, &texw, &texh);
    }

    for(int  i = 0; i < num_vertices; i++) {
        Positions[i] = vertices[i].position;
        Colors[i] = vertices[i].colour;
        if (sdl_texture) {
            TexCoords[i].x = vertices[i].tex_coord.x * texw;
            TexCoords[i].y = vertices[i].tex_coord.y * texh;
        }
        else TexCoords[i] = vertices[i].tex_coord;
    };

    unsigned short newIndicies[num_indices];
    for (int i = 0; i < num_indices; i++) {
      newIndicies[i] = (unsigned short) indices[i];
    }

    render_data.glEnableClientState(GL_VERTEX_ARRAY);
    render_data.glEnableClientState(GL_COLOR_ARRAY);
    render_data.glVertexPointer(2, GL_FLOAT, 0, &Positions[0]);
    render_data.glColorPointer(4, GL_UNSIGNED_BYTE, 0, &Colors[0]);
    render_data.glTexCoordPointer(2, GL_FLOAT, 0, &TexCoords[0]);

    render_data.glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE);
    render_data.glEnable(GL_BLEND);
    render_data.glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
    render_data.glDrawElements(GL_TRIANGLES, num_indices, GL_UNSIGNED_SHORT, newIndicies);
    render_data.glDisableClientState(GL_VERTEX_ARRAY);
    render_data.glDisableClientState(GL_COLOR_ARRAY);
    render_data.glDisableClientState(GL_TEXTURE_COORD_ARRAY);

    if (sdl_texture) {
        SDL_GL_UnbindTexture(sdl_texture);
        render_data.glDisableClientState(GL_TEXTURE_COORD_ARRAY);
    }

    render_data.glColor4f(1.0, 1.0, 1.0, 1.0);
    render_data.glPopMatrix();
    /* Reset blending and draw a fake point just outside the screen to let SDL know that it needs to reset its state in case it wants to render a texture */
    render_data.glDisable(GL_BLEND);
    SDL_SetRenderDrawBlendMode(renderer, SDL_BLENDMODE_NONE);
    SDL_RenderDrawPoint(renderer, -1, -1);
void RocketSDLRenderInterfaceOpenGLES2::RenderGeometry(Rocket::Core::Vertex* vertices, int num_vertices, int* indices, int num_indices, const Rocket::Core::TextureHandle texture, const Rocket::Core::Vector2f& translation) {
SDL_Texture* sdl_texture = NULL;
    if(texture) render_data.glUseProgram(program_texture_id);
    else render_data.glUseProgram(program_color_id);
    int width, height;
    SDL_Rect rvp;
    SDL_RenderGetViewport(renderer, &rvp);

    GLfloat projection[4][4];

    // Prepare an orthographic projection
    projection[0][0] = 2.0f / rvp.w;
    projection[0][1] = 0.0f;
    projection[0][2] = 0.0f;
    projection[0][3] = 0.0f;
    projection[1][0] = 0.0f;
    //if (renderer->target) {
    //    projection[1][1] = 2.0f / height;
    //} else {
        projection[1][1] = -2.0f / rvp.h;
    //}
    projection[1][2] = 0.0f;
    projection[1][3] = 0.0f;
    projection[2][0] = 0.0f;
    projection[2][1] = 0.0f;
    projection[2][2] = 0.0f;
    projection[2][3] = 0.0f;
    projection[3][0] = -1.0f;
    //if (renderer->target) {
    //    projection[3][1] = -1.0f;
    //} else {
        projection[3][1] = 1.0f;
    //}
    projection[3][2] = 0.0f;
    projection[3][3] = 1.0f;

    // Set the projection matrix
    if (texture) {
        render_data.glUniformMatrix4fv(u_texture_projection, 1, GL_FALSE, (GLfloat *)projection);
        render_data.glUniform2f(u_texture_translation, translation.x, translation.y);
    }
    else {
        render_data.glUniformMatrix4fv(u_color_projection, 1, GL_FALSE, (GLfloat *)projection);
        render_data.glUniform2f(u_color_translation, translation.x, translation.y);
    }

    render_data.glEnable(GL_BLEND);
    render_data.glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);

    float texw, texh;

    unsigned short newIndicies[num_indices];
    for (int i = 0; i < num_indices; i++)
    {
      newIndicies[i] = (unsigned short) indices[i];
    }

    glVertexAttribPointer(ROCKETGLUE_ATTRIBUTE_POSITION, 2, GL_FLOAT, GL_FALSE, sizeof(Rocket::Core::Vertex), &vertices[0].position);
    glVertexAttribPointer(ROCKETGLUE_ATTRIBUTE_COLOR, 4, GL_UNSIGNED_BYTE, GL_TRUE, sizeof(Rocket::Core::Vertex), &vertices[0].colour);
    render_data.glEnableVertexAttribArray(ROCKETGLUE_ATTRIBUTE_POSITION);
    render_data.glEnableVertexAttribArray(ROCKETGLUE_ATTRIBUTE_TEXCOORD);
    render_data.glEnableVertexAttribArray(ROCKETGLUE_ATTRIBUTE_COLOR);

    if(texture) {
        sdl_texture = (SDL_Texture *) texture;
        SDL_GL_BindTexture(sdl_texture, &texw, &texh);
        render_data.glUniform1i(u_texture, 0);
        glVertexAttribPointer(ROCKETGLUE_ATTRIBUTE_TEXCOORD, 2, GL_FLOAT, GL_FALSE, sizeof(Rocket::Core::Vertex), &vertices[0].tex_coord);
    }
    else {
        render_data.glActiveTexture(GL_TEXTURE0);
        render_data.glDisable(GL_TEXTURE_2D);
        render_data.glDisableVertexAttribArray(ROCKETGLUE_ATTRIBUTE_TEXCOORD);
    }

    render_data.glDrawElements(GL_TRIANGLES, num_indices, GL_UNSIGNED_SHORT, newIndicies);

    /* We can disable ROCKETGLUE_ATTRIBUTE_COLOR (2) safely as SDL will reenable the vertex attrib 2 if it is required */
    render_data.glDisableVertexAttribArray(ROCKETGLUE_ATTRIBUTE_COLOR);

    /* Leave ROCKETGLUE_ATTRIBUTE_POSITION (0) and ROCKETGLUE_ATTRIBUTE_TEXCOORD (1) enabled for compatibility with SDL which
       doesn't re enable them when you call RenderCopy/Ex */
    if(sdl_texture) SDL_GL_UnbindTexture(sdl_texture);
    else render_data.glEnableVertexAttribArray(ROCKETGLUE_ATTRIBUTE_TEXCOORD);

    /* Reset blending and draw a fake point just outside the screen to let SDL know that it needs to reset its state in case it wants to render a texture */
    render_data.glDisable(GL_BLEND);
    SDL_SetRenderDrawBlendMode(renderer, SDL_BLENDMODE_NONE);
    SDL_RenderDrawPoint(renderer, -1, -1);
}

As you can see, there’s a fair amount of gymnastics involved in getting SDL to a state where it can be made to cooperate with libRocket. The full source code (including texture loading and input translation) is available in the Rocket* files in the Ignifuga source code. The libRocket fork Ignifuga uses is hosted at Github.
So far libRocket has proven to be very easy to use and more so because on top of it Ignifuga adds pQuery, a jQuery inspired set of utility functions, which makes animating libRocket a blast! (pun intended)

Programming, Python, Ubuntu

6 comments

In the last few months I’ve been hard at work on my Python/Cython based game engine, Ignifuga, which I’m developing alongside a mysterious game project (by the way, do you know who is the gaucho?). The rationale behind my choice of language had several aspects to it. I was already familiar with Python, it’s a widespread language consistently in the top 5 of the most popular languages with a huge community behind it, and I felt there was no engine that did what I wanted my engine to have, that is be cross platform (at least Win/OS X/Linux and iOS / Android), data driven, simple to experiment and iterate changes. The cautionary tales about Python speed were written all across the web, but I figured that the Cython/Python combo will let me develop the engine first, and fine tune it later.

A couple of weeks ago I stumbled upon Haxe and more importantly across Haxe NME. Basically Haxe NME provides most of what I wanted to do, but it is centered on the Haxe language, which is very similar to Action Script 3. The Haxe compiler outputs (among others) CPP and Javascript code, and being mostly statically typed, the apps it produces are very fast, as proven by this benchmark, a variation of the BunnyMark

Those numbers wet my appetite to see what sort of results I would get with my own engine, so I made my version in pure Python in a couple of hours. Since Ignifuga is entity/component based, I created a number of entities with a sprite component attached, and in every update loop I moved them around. The results came, on my Ubuntu 12.04 i5 2500K system with a Nvidia GTX 570 and 302.17 binary drivers, at 30 FPS I got 1200 bunnies. So, what I had basically built was a low end smartphone emulator, seeing that I was getting worse figures than the Haxe BunnyMark ran on a LG Optimus One.

Remember when I said that I chose Python/Cython for the ease of implementation while retaining the opportunity to improve performance later on…well, it felt “later on” had arrived. Ignifuga has two big loops ran in succession, the entities update loop and the renderer loop. The first one is ran using greenlets, which is a sort of cooperative threading, and lets entities and components update themselves. The renderer loop iterates over the existing sprite components, intersecting their position with the current visible area, and those that fall in that zone are renderer to the screen using SDL. Almost every operation I just mentioned was done in Python, so this was a good starting point as any to start improving. To confirm my suspicions, I activated the basic profiling abilities that Python and Cython provide, and indeed I confirmed that every rendering loop was taking about 33 ms with 1200 bunnies on screen (that is, the entire alloted time for the frame).

You can see what the renderer update loop looked like before the optimization here. Thanks to Cython’s seamless integration with CPP, I quickly replaced all the Python dict/list iteration based code (which I believe was what was eating up most of the CPU time), with fast STL containers. I also added some extra intelligence to the code, to avoid intersecting rectangles excessively when the corresponding sprites hadn’t moved (which is useless for the BunnyMark as all sprites move every frame, but it’s useful for real apps). With this optimizations in place, I ran the benchmark again…and got some glorious 1400 bunnies on screen.

Sigh…time to profile the code again. This time, the renderer loop was at less than 1ms per frame, and the update loop was now eating all the CPU bandwidth (we are talking about A LOT of CPU bandwith!). Time to fix up the update loop using the same strategy, I removed dicts/lists and added STL containers, I called the Greenlet functions directly, etc. The benchmark now gave me an overwhelming 1600 bunnies…but now the CPU cycles went almost entirely to the bunnies.py update loop itself.

Being the competitive type of person that I am, I could not stop there. So, I made a Cython version of the Bunnymark. And to top that, I cheated. Yes, I confess…I cheated. As the Haxe NME manipulates the sprites positions directly, I decided to cut through all of the nice scaffolding I had built to make development easier and cleaner (each entity and its components have their own update loop, it’s all neat and cleanly separated, etc). This is the result, it creates a single entity, and then it clones the sprite component as many times as needed, and then it moves thos sprites around directly without bothering to go through the “proper channels”. The result now looked promising, 57.000 bunnies (versus the 1600 I had with the pure Python benchmark).

I then tested this benchmark on my other systems. On an ASUS Transformer TF101 I’m getting 2600 bunnies at 30 FPS (Haxe NME gets 4000), on an iPad 2 1600 bunnies at 30 FPS (extrapolating from the results published Haxe NME gets about 6000). The reason for this difference may lie in several places, I’m still going through some parts of Python code in time sensitive parts of my code, Ignifuga being an engine, I may have more scaffolding running at any given time than what the Haxe version has, etc. But overall I feel this is a very good result (specially seeing what I started with).

A nice advantage Cython includes is the ability to apply seamless parallelization where OpenMP is available (which seems to be Linux only). I’ve added a bit of this on my renderer loop (the part which intersects the sprite rectangles with the screen rectangles), and on the bunnies update loop (this is the OpenMP version of the Bunnymark). As a result, I got 60000 bunnies on my i5 2500k Linux system (3000 more than the non parallel version), to understand why I did not get more, it’s time to see a few profile dumps…

Ubuntu 12.04, i5 2500k, GTX 570, non parallel (1 CPU at 100% load), 57k bunnies

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   73.544   73.544  146.260  146.260 GameLoop.pyx:27(run)
     2223   62.774    0.028   62.838    0.028 Renderer.pyx:280(update)
     2271    9.398    0.004    9.796    0.004 GameLoopBase.pyx:263(_doSwitch)
        1    0.122    0.122    0.122    0.122 SDL.pyx:13(initializeSDL)
        1    0.120    0.120    0.120    0.120 Renderer.pyx:43(__init__)
      569    0.106    0.000    0.107    0.000 Canvas.pyx:228(text)
        3    0.065    0.022    0.065    0.022 Canvas.pyx:26(__init__)
     2223    0.064    0.000    0.064    0.000 Renderer.pyx:664(flip)
    57771    0.047    0.000    0.054    0.000 Renderer.pyx:341(addSprite)
      570    0.030    0.000    0.030    0.000 Renderer.pyx:371(removeSprite)
        1    0.018    0.018  146.524  146.524 {bunniesx.run}
   172229    0.018    0.000    0.018    0.000 Renderer.pyx:437(spriteDst)
   172229    0.015    0.000    0.015    0.000 Renderer.pyx:441(spriteRot)
   172229    0.014    0.000    0.014    0.000 Renderer.pyx:445(spriteColor)
     2268    0.011    0.000    9.872    0.004 GameLoopBase.pyx:291(_processTask)
    57771    0.008    0.000    0.008    0.000 Renderer.pyx:316(_indexSprite)
     2223    0.005    0.000    9.877    0.004 GameLoopBase.pyx:169(update)

Ubuntu 12.04, i5 2500k, GTX 570, OpenMP parallel (4 CPUs at about 25% load each), 60k bunnies

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   63.751   63.751  118.381  118.381 GameLoop.pyx:27(run)
     1763   48.222    0.027   48.268    0.027 Renderer.pyx:280(update)
     1810    5.868    0.003    6.278    0.003 GameLoopBase.pyx:263(_doSwitch)
        1    0.126    0.126    0.127    0.127 Renderer.pyx:43(__init__)
      593    0.112    0.000    0.113    0.000 Canvas.pyx:228(text)
        1    0.106    0.106    0.106    0.106 SDL.pyx:13(initializeSDL)
        3    0.067    0.022    0.067    0.022 Canvas.pyx:26(__init__)
    60195    0.048    0.000    0.056    0.000 Renderer.pyx:341(addSprite)
     1763    0.046    0.000    0.046    0.000 Renderer.pyx:664(flip)
      594    0.032    0.000    0.032    0.000 Renderer.pyx:371(removeSprite)
        1    0.019    0.019  118.637  118.637 {bunniesp.run}
   179451    0.017    0.000    0.017    0.000 Renderer.pyx:437(spriteDst)
   179451    0.016    0.000    0.016    0.000 Renderer.pyx:441(spriteRot)
   179451    0.014    0.000    0.014    0.000 Renderer.pyx:445(spriteColor)
     1807    0.011    0.000    6.356    0.004 GameLoopBase.pyx:291(_processTask)
    60195    0.008    0.000    0.008    0.000 Renderer.pyx:316(_indexSprite)
     1763    0.004    0.000    6.360    0.004 GameLoopBase.pyx:169(update)

As you can see, in both cases the renderer loop still eats up most of the frame time (27/28 ms out of the 33ms available for 30 FPS). There’s a key renderer detail to consider: the OpenGL function calls need to be serialized, they can’t be parallelized, so we’ve basically hit a bottleneck, the one brought forth the graphics card/driver performance. As a sidenote, this is the profile output of a much slower computer running Windows with a GTX 285 GPU and Nvidia drivers:

Windows 7, Q6600, GTX 285, no parallel, 61k bunnies (using SDL’s OpenGL backend)

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   50.177   50.177   90.285   90.285 GameLoop.pyx:27(run)
     1466   30.128    0.021   30.222    0.021 Renderer.pyx:280(update)
     1514    9.110    0.006    9.728    0.006 GameLoopBase.pyx:263(_doSwitch)
        1    0.225    0.225    0.225    0.225 Renderer.pyx:43(__init__)
      583    0.203    0.000    0.204    0.000 Canvas.pyx:228(text)
        3    0.140    0.047    0.140    0.047 Canvas.pyx:26(__init__)
     1466    0.094    0.000    0.094    0.000 Renderer.pyx:664(flip)
   176443    0.075    0.000    0.096    0.000 Renderer.pyx:437(spriteDst)
   176443    0.071    0.000    0.092    0.000 Renderer.pyx:441(spriteRot)
   176443    0.066    0.000    0.092    0.000 Renderer.pyx:445(spriteColor)
        1    0.061    0.061    0.061    0.061 SDL.pyx:20(terminateSDL)
    59185    0.027    0.000    0.096    0.000 Renderer.pyx:341(addSprite)
        1    0.023    0.023   90.644   90.644 {bunniesx.run}
      112    0.020    0.000    0.020    0.000 Log.pyx:46(log)
     1511    0.010    0.000    9.879    0.007 GameLoopBase.pyx:291(_processTask)

    59185    0.009    0.000    0.009    0.000 Renderer.pyx:316(_indexSprite)
     1466    0.004    0.000    9.882    0.007 GameLoopBase.pyx:169(update)

This clearly shows the sorry state of nVidia’s binary drivers for Linux. The Windows version of the drivers are much faster, evidenced by the fact that on a much slower computer, the non parallelized renderer loop takes 21ms and it shows 1K more bunnies than the parallel version under Linux. In this slower system, the CPU is now the limiting factor at 61k bunnies. On my faster i5-2500K system, using Windows (non parallel version) I get 127300 bunnies (more than double the amount of bunnies using the exact same hardware). I don’t have the profiling output for that one, but I believe the limit is again on the CPU.

In the near future I plan on experimenting with ZeroMQ for paralellization, as OpenMP’s support in Cython is seamless and very neat but it suffers from a two big problems…there’s very little support for it outside of Linux, and the runtime library is GPL licensed.

Android, Programming, Python, Ubuntu

No comments

So, the cat, or should I say the chicken, is out of the bag! I’ve opened up the website for my pet project, a Python/SDL based 2D game engine called “Ignifuga”. You can check it out here.

Ubuntu

No comments

I’ve been a power Kubuntu user (I use it in my main system for everyday work) since 8.04. Every update to a newer version has had its fair share of minor issues and some not so minor, but all in all I haven’t hit a major roadblock (that I can remember), even when KDE switched from the 3.x to 4.x series. That is, until now. I’m really surprised about the number of major issues I’ve hit on this release, and a sad low point in the history of Ubuntu. After migrating with Muon, I rebooted and…

Boot takes a looooong time, no console or garbage in the console

Nvidia video cards don’t deal well with the vt.handoff parameter that Ubuntu adds by default to the kernel boot line in Grub. The upgrade process puts a /etc/grub.d/10_linux with this parameter, that you have to remove to get a useful console. Solution:

  • Select recovery mode in Grub, press e on that option, look for the line with vt.handoff=7 and remove it, then press CTRL-X.
  • Select root console
  • mount -o rw,remount /
  • Edit /etc/grub.d/10_linux, remove vt.handoff=7
  • update-grub
  • reboot
Now you can see the console, it still takes a looooong time, “Waiting for network connection”

It seems Ubuntu is now more serious about network interfaces and their availability/readiness. Too bad they didn’t bother asking the user if he wants this or issuing a big warning during the upgrade process. Anyway, if you have interfaces listed in /etc/network/interfaces that don’t exist in your system or that can’t get an IP quick enough, the system will keep you waiting without telling you what it’s really waiting for. Most solutions I’ve found online point to yet another random change in 11.10 where they moved /var/run to /run and /var/lock to /run/lock (I guess they felt like keeping things fresh by doing pointless changes, or maybe I’m just bitter because of the many hours I spent trying to figure out the problem). Solution:

  • Boot in recovery mode
  • mount -o rw,remount /
  • Edit /etc/network/interfaces, comment out all mentions to network interfaces that you don’t actually have or that are not connected
  • reboot
Nvidia propietary driver is no longer working, no X server

I have an Nvidia card configured with propietary drivers and so the X server depends on them. The upgrade process removed the Ubuntu X-Swat PPA, so the Nvidia driver didn’t update. As the upgrade process installs a new kernel, some conflict happens (the details of which I don’t care enough to find out, but be certain that it’s a problem), and you are left out the X environment with no warning of any kind. Solution:

  • Boot in recovery mode, go to root console.
  • mount -o rw,remount /
  • Edit /etc/apt/sources.list.d/ubuntu-x-swat (something).list, enable the repository again
  • apt-get update;apt-get –reinstall install nvidia-current
  • reboot
Now X starts, but KDE complains of dbus being not available, and ask you to run qdbus.
For some reason, the package qdbus is not installed in the upgrade process. Without it, guess what…KDE doesn’t start!
Solution:
  • Boot in recovery mode, go to root console.
  • mount -o rw,remount /
  • apt-get install qdbus
At this point you are seriously considering installing Windows 95 and be done with it, but the problems don’t stop there…
No sound
My system has two sound cards, the motherboard embedded one by Realtek and the HDMI sound card by Nvidia. Using pavucontrol to set the correct default was required (this was working properly before the upgrade)
KMail 2 has a lot of issues and is slow
Well, at least Kubuntu has a nice page covering most of the problems you may hit
PostgreSQL 8.4 has 100% CPU usage, PostgreSQL 9 is installed in parallel, Akonadi uses 8.4 instead of 9
Once you get KMail 2 working, it seems the data backend Akonadi forces PostgreSQL 8.4 to go into crazy mode as it just jumps to 100% CPU usage and never lets go no matter what. To make things worse, the upgrade process has installed PostgreSQL 9 in your system, but didn’t migrate the data. If you do so manually (as I did to see if the migration solved the CPU usage problem), Akonadi will now complain that it doesn’t find the 8.4 server binary (which of course you had just removed…). Sigh.

Solution:

  • Edit /etc/postgresql/main/postgresql.conf, make sure port=5432 (I guess the upgrade process had configured it on 5433 because 5432 was used by PostgreSQL 8.4)
  • Apply this solution otherwise PostgreSQL 9.1 probably won’t start automatically after boot (though it will if you do service postgresql start afterwards…for reasons unknown to me!)
  • sudo service postgresql restart
  • Edit ~/.config/akonadi/akonadiserverrc and set the correct path (usually involves changing 8.4 for 9.1 in the PGSQL paths)
At least the Postgres high CPU usage problem seems to happens less often, though it’s not completely solved…anyway, I think I’ll just use Gmail via the web interface until this issue is solved.
Skype doesn’t load
I had to follow the instructions here, to install 32 bit libraries that I thought I already had in place as it was working in Ubuntu Natty. This was not enough however, as some manual file deletion was needed as explained here.

Android, Programming

No comments

Update (Oct 9th, 2012): The third message in this thread explains a simpler alternative to what’s explained below. I haven’t tried it myself but it seems cleaner and easier to implement, so you should probably try that one first.

Update 2 (Feb 21st, 2013): I’m now using the simplified method mentioned above in my engine. I copy the org.libsdl.app.SDLActivity into my project and inherit from it from a class with the package and class name that I want to use in my app.

The only change I had to do on the standard SDLActivity.java is to modify the static section of this file to load the rest of the remaining libraries I require, doing this in your inherited class doesn’t quite work depending on how each library depends on the others.

The method stated below is still useful if your app does something that the default SDLActivity can’t really handle, such as a live wallpaper.

The Android SDL backend works great, it comes with a full skeleton project that you can use to get up to speed quickly. There is however one minor problem, everything in said project is hardwired to use the “org.libsdl.app” app name, which is all fine and dandy until you want to make a real world app with your own name…and keep using the SDL source with as little modifications as possible to benefit from the latest and greatest modifications provided by the open source community.

So, it’s time to take out the glue gun and put it to good use. What I do is deploy a little piece of code that I call jni_glue.cpp and that goes into jni/src/jni_glue.cpp. In my case the app name is “com.mdqinc.dp”…

#include "jni.h"
#include "SDL_config.h"
#include "SDL_stdinc.h"

//static float fLastAccelerometer[3];
extern "C" {
    void Java_org_libsdl_app_SDLActivity_nativeRunAudioThread(JNIEnv* env, jclass cls);
    void Java_org_libsdl_app_SDLActivity_onNativeResize(
                                    JNIEnv* env, jclass jcls,
                                    jint width, jint height, jint format);
    void Java_org_libsdl_app_SDLActivity_onNativeKeyDown(
                                    JNIEnv* env, jclass jcls, jint keycode);
    void Java_org_libsdl_app_SDLActivity_onNativeKeyUp(
                                    JNIEnv* env, jclass jcls, jint keycode);
    void Java_org_libsdl_app_SDLActivity_onNativeTouch(
                                    JNIEnv* env, jclass jcls,
                                    jint touch_device_id_in, jint pointer_finger_id_in,
                                    jint action, jfloat x, jfloat y, jfloat p);
    void Java_org_libsdl_app_SDLActivity_onNativeAccel(
                                    JNIEnv* env, jclass jcls,
                                    jfloat x, jfloat y, jfloat z);
    void Java_org_libsdl_app_SDLActivity_nativeQuit(
                                    JNIEnv* env, jclass cls);
    void Java_org_libsdl_app_SDLActivity_nativeRunAudioThread(
                                    JNIEnv* env, jclass cls);
    void Java_org_libsdl_app_SDLActivity_nativeInit(JNIEnv* env, jclass cls, jobject obj);
};

// Resize
extern "C" void Java_com_mdqinc_dp_SDLActivity_onNativeResize(
                                    JNIEnv* env, jclass jcls,
                                    jint width, jint height, jint format)
{
    Java_org_libsdl_app_SDLActivity_onNativeResize(env, jcls, width, height, format);
}

// Keydown
extern "C" void Java_com_mdqinc_dp_SDLActivity_onNativeKeyDown(
                                    JNIEnv* env, jclass jcls, jint keycode)
{
    Java_org_libsdl_app_SDLActivity_onNativeKeyDown(env, jcls, keycode);
}

// Keyup
extern "C" void Java_com_mdqinc_dp_SDLActivity_onNativeKeyUp(
                                    JNIEnv* env, jclass jcls, jint keycode)
{
    Java_org_libsdl_app_SDLActivity_onNativeKeyUp(env, jcls, keycode);
}

// Touch
extern "C" void Java_com_mdqinc_dp_SDLActivity_onNativeTouch(
                                    JNIEnv* env, jclass jcls,
                                    jint touch_device_id_in, jint pointer_finger_id_in,
                                    jint action, jfloat x, jfloat y, jfloat p)
{
    Java_org_libsdl_app_SDLActivity_onNativeTouch(env, jcls, touch_device_id_in, pointer_finger_id_in, action, x, y, p);
}

// Accelerometer
extern "C" void Java_com_mdqinc_dp_SDLActivity_onNativeAccel(
                                    JNIEnv* env, jclass jcls,
                                    jfloat x, jfloat y, jfloat z)
{
     Java_org_libsdl_app_SDLActivity_onNativeAccel(env, jcls, x, y, z);
}

// Quit
extern "C" void Java_com_mdqinc_dp_SDLActivity_nativeQuit(
                                    JNIEnv* env, jclass cls)
{
    Java_org_libsdl_app_SDLActivity_nativeQuit(env, cls);
}

extern "C" void Java_com_mdqinc_dp_SDLActivity_nativeRunAudioThread(
                                    JNIEnv* env, jclass cls)
{
    Java_org_libsdl_app_SDLActivity_nativeRunAudioThread(env, cls);
}

extern "C" void Java_com_mdqinc_dp_SDLActivity_nativeInit(JNIEnv* env, jclass cls, jobject obj) {
    Java_org_libsdl_app_SDLActivity_nativeInit(env, cls, obj);
}

This glue code should be paired with a Java activity file based on the one SDL ships, but you need to modify the line that says “package org.libsdl.app;” for a name that suits your needs (and that matches the glue code function names posted above). The same modification needs to be applied to the AndroidManifest.xml.

Besides these changes you need to take care of a final minor adjustment, you have to place your activity file in the right location. It will no longer go in src/org/libsdl/app/SDLActivity.java but rather (for example, in my case) in src/org/mdqinc/dp/SDLActivity.java.

You can certainly achieve the same result by modifying the SDL source code directly, but you’ll find that updating it later is a hassle. This approach I’m describing stands separate from the actual SDL code, so you can update it without worries. The only thing you may want to keep an eye on is changes in the SDLActivity.java file, but in case that’s updated you only need to change a single line with the package name at the top.

Programming, Python

No comments

The most famous Python bindings for SDL is probably the PyGame project, but it’s oriented to the 1.2.x versions of the multiplatform library. There’s also PySDL by Albert Zeyer, which are automatically created from the SDL headers using a generator.

I tried to do the same for Cython, using an automated generator to parse header files and output a Cython sxd file. Sadly the tools for that are a bit green and they don’t quite work yet, so I’ve made a Cython header file handcrafted from SDL 1.3 headers containing the functions I need (There’s SDL, SDL_image and SDL_ttf functions in there). Based on it and the SDL source code it’s very easy to expand it, so contributions are most welcome.

Without further ado, SDL.pxd:

cdef extern from "SDL.h":
    ctypedef unsigned char Uint8
    ctypedef unsigned long Uint32
    ctypedef unsigned long long Uint64
    ctypedef signed long long Sint64
    ctypedef signed short Sint16
    ctypedef unsigned short Uint16

    ctypedef enum:
        SDL_PIXELFORMAT_ARGB8888

    ctypedef enum SDL_BlendMode:
        SDL_BLENDMODE_NONE = 0x00000000
        SDL_BLENDMODE_BLEND = 0x00000001
        SDL_BLENDMODE_ADD = 0x00000002
        SDL_BLENDMODE_MOD = 0x00000004

    ctypedef enum SDL_TextureAccess:
        SDL_TEXTUREACCESS_STATIC
        SDL_TEXTUREACCESS_STREAMING
        SDL_TEXTUREACCESS_TARGET

    ctypedef enum SDL_RendererFlags:
        SDL_RENDERER_SOFTWARE = 0x00000001
        SDL_RENDERER_ACCELERATED = 0x00000002
        SDL_RENDERER_PRESENTVSYNC = 0x00000004

    ctypedef enum SDL_bool:
        SDL_FALSE = 0
        SDL_TRUE = 1

    cdef struct SDL_Rect:
        int x, y
        int w, h

    ctypedef struct SDL_Point:
        int x, y

    cdef struct SDL_Color:
        Uint8 r
        Uint8 g
        Uint8 b
        Uint8 unused

    cdef struct SDL_Palette:
        int ncolors
        SDL_Color *colors
        Uint32 version
        int refcount

    cdef struct SDL_PixelFormat:
        Uint32 format
        SDL_Palette *palette
        Uint8 BitsPerPixel
        Uint8 BytesPerPixel
        Uint8 padding[2]
        Uint32 Rmask
        Uint32 Gmask
        Uint32 Bmask
        Uint32 Amask
        Uint8 Rloss
        Uint8 Gloss
        Uint8 Bloss
        Uint8 Aloss
        Uint8 Rshift
        Uint8 Gshift
        Uint8 Bshift
        Uint8 Ashift
        int refcount
        SDL_PixelFormat *next

    cdef struct SDL_BlitMap

    cdef struct SDL_Surface:
        Uint32 flags
        SDL_PixelFormat *format
        int w, h
        int pitch
        void *pixels
        void *userdata
        int locked
        void *lock_data
        SDL_Rect clip_rect
        SDL_BlitMap *map
        int refcount

    ctypedef enum SDL_EventType:
        SDL_FIRSTEVENT     = 0,
        SDL_QUIT           = 0x100
        SDL_WINDOWEVENT    = 0x200
        SDL_SYSWMEVENT
        SDL_KEYDOWN        = 0x300
        SDL_KEYUP
        SDL_TEXTEDITING
        SDL_TEXTINPUT
        SDL_MOUSEMOTION    = 0x400
        SDL_MOUSEBUTTONDOWN
        SDL_MOUSEBUTTONUP
        SDL_MOUSEWHEEL
        SDL_INPUTMOTION    = 0x500
        SDL_INPUTBUTTONDOWN
        SDL_INPUTBUTTONUP
        SDL_INPUTWHEEL
        SDL_INPUTPROXIMITYIN
        SDL_INPUTPROXIMITYOUT
        SDL_JOYAXISMOTION  = 0x600
        SDL_JOYBALLMOTION
        SDL_JOYHATMOTION
        SDL_JOYBUTTONDOWN
        SDL_JOYBUTTONUP
        SDL_FINGERDOWN      = 0x700
        SDL_FINGERUP
        SDL_FINGERMOTION
        SDL_TOUCHBUTTONDOWN
        SDL_TOUCHBUTTONUP
        SDL_DOLLARGESTURE   = 0x800
        SDL_DOLLARRECORD
        SDL_MULTIGESTURE
        SDL_CLIPBOARDUPDATE = 0x900
        SDL_EVENT_COMPAT1 = 0x7000
        SDL_EVENT_COMPAT2
        SDL_EVENT_COMPAT3
        SDL_USEREVENT    = 0x8000
        SDL_LASTEVENT    = 0xFFFF

    ctypedef enum SDL_WindowEventID:
        SDL_WINDOWEVENT_NONE           #< Never used */
        SDL_WINDOWEVENT_SHOWN          #< Window has been shown */
        SDL_WINDOWEVENT_HIDDEN         #< Window has been hidden */
        SDL_WINDOWEVENT_EXPOSED        #< Window has been exposed and should be
                                        #     redrawn */
        SDL_WINDOWEVENT_MOVED          #< Window has been moved to data1, data2
                                        # */
        SDL_WINDOWEVENT_RESIZED        #< Window has been resized to data1xdata2 */
        SDL_WINDOWEVENT_SIZE_CHANGED   #< The window size has changed, either as a result of an API call or through the system or user changing the window size. */
        SDL_WINDOWEVENT_MINIMIZED      #< Window has been minimized */
        SDL_WINDOWEVENT_MAXIMIZED      #< Window has been maximized */
        SDL_WINDOWEVENT_RESTORED       #< Window has been restored to normal size
                                        # and position */
        SDL_WINDOWEVENT_ENTER          #< Window has gained mouse focus */
        SDL_WINDOWEVENT_LEAVE          #< Window has lost mouse focus */
        SDL_WINDOWEVENT_FOCUS_GAINED   #< Window has gained keyboard focus */
        SDL_WINDOWEVENT_FOCUS_LOST     #< Window has lost keyboard focus */
        SDL_WINDOWEVENT_CLOSE           #< The window manager requests that the
                                        # window be closed */

    ctypedef enum SDL_WindowFlags:
        SDL_WINDOW_FULLSCREEN = 0x00000001
        SDL_WINDOW_OPENGL = 0x00000002
        SDL_WINDOW_SHOWN = 0x00000004
        SDL_WINDOW_HIDDEN = 0x00000008
        SDL_WINDOW_BORDERLESS = 0x00000010
        SDL_WINDOW_RESIZABLE = 0x00000020
        SDL_WINDOW_MINIMIZED = 0x00000040
        SDL_WINDOW_MAXIMIZED = 0x00000080
        SDL_WINDOW_INPUT_GRABBED = 0x00000100
        SDL_WINDOW_INPUT_FOCUS = 0x00000200
        SDL_WINDOW_MOUSE_FOCUS = 0x00000400
        SDL_WINDOW_FOREIGN = 0x00000800

    ctypedef enum SDL_RendererFlip:
        SDL_FLIP_NONE = 0x00000000
        SDL_FLIP_HORIZONTAL = 0x00000001
        SDL_FLIP_VERTICAL = 0x00000002

    cdef struct SDL_MouseMotionEvent:
        Uint32 type
        Uint32 windowID
        Uint8 state
        Uint8 padding1
        Uint8 padding2
        Uint8 padding3
        int x
        int y
        int xrel
        int yrel

    cdef struct SDL_MouseButtonEvent:
        Uint32 type
        Uint32 windowID
        Uint8 button
        Uint8 state
        Uint8 padding1
        Uint8 padding2
        int x
        int y

    cdef struct SDL_WindowEvent:
        Uint32 type
        Uint32 windowID
        Uint8 event
        Uint8 padding1
        Uint8 padding2
        Uint8 padding3
        int data1
        int data2

    ctypedef Sint64 SDL_TouchID
    ctypedef Sint64 SDL_FingerID

    cdef struct SDL_TouchFingerEvent:
        Uint32 type
        Uint32 windowID
        SDL_TouchID touchId
        SDL_FingerID fingerId
        Uint8 state
        Uint8 padding1
        Uint8 padding2
        Uint8 padding3
        Uint16 x
        Uint16 y
        Sint16 dx
        Sint16 dy
        Uint16 pressure

    cdef struct SDL_KeyboardEvent:
        pass
    cdef struct SDL_TextEditingEvent:
        pass
    cdef struct SDL_TextInputEvent:
        pass
    cdef struct SDL_MouseWheelEvent:
        Uint32 type
        Uint32 windowID
        int x
        int y

    cdef struct SDL_JoyAxisEvent:
        pass
    cdef struct SDL_JoyBallEvent:
        pass
    cdef struct SDL_JoyHatEvent:
        pass
    cdef struct SDL_JoyButtonEvent:
        pass
    cdef struct SDL_QuitEvent:
        pass
    cdef struct SDL_UserEvent:
        pass
    cdef struct SDL_SysWMEvent:
        pass
    cdef struct SDL_TouchFingerEvent:
        pass
    cdef struct SDL_TouchButtonEvent:
        pass
    cdef struct SDL_MultiGestureEvent:
        pass
    cdef struct SDL_DollarGestureEvent:
        pass

    cdef union SDL_Event:
        Uint32 type
        SDL_WindowEvent window
        SDL_KeyboardEvent key
        SDL_TextEditingEvent edit
        SDL_TextInputEvent text
        SDL_MouseMotionEvent motion
        SDL_MouseButtonEvent button
        SDL_MouseWheelEvent wheel
        SDL_JoyAxisEvent jaxis
        SDL_JoyBallEvent jball
        SDL_JoyHatEvent jhat
        SDL_JoyButtonEvent jbutton
        SDL_QuitEvent quit
        SDL_UserEvent user
        SDL_SysWMEvent syswm
        SDL_TouchFingerEvent tfinger
        SDL_TouchButtonEvent tbutton
        SDL_MultiGestureEvent mgesture
        SDL_DollarGestureEvent dgesture

    cdef struct SDL_RendererInfo:
        char *name
        Uint32 flags
        Uint32 num_texture_formats
        Uint32 texture_formats[16]
        int max_texture_width
        int max_texture_height

    ctypedef struct SDL_Texture
    ctypedef struct SDL_Renderer
    ctypedef struct SDL_Window
    ctypedef struct SDL_DisplayMode:
        Uint32 format
        int w
        int h
        int refresh_rate
        void *driverdata

    cdef struct SDL_RWops:
        long (* seek) (SDL_RWops * context, long offset,int whence)
        size_t(* read) ( SDL_RWops * context, void *ptr, size_t size, size_t maxnum)
        size_t(* write) (SDL_RWops * context, void *ptr,size_t size, size_t num)
        int (* close) (SDL_RWops * context)

    cdef SDL_Renderer * SDL_CreateRenderer(SDL_Window * window, int index, Uint32 flags)
    cdef SDL_Texture * SDL_CreateTexture(SDL_Renderer * renderer, Uint32 format, int access, int w, int h)
    cdef SDL_Texture * SDL_CreateTextureFromSurface(SDL_Renderer * renderer, SDL_Surface * surface)
    cdef SDL_Surface * SDL_CreateRGBSurface(Uint32 flags, int width, int height, int depth, Uint32 Rmask, Uint32 Gmask, Uint32 Bmask, Uint32 Amask)
    cdef int SDL_RenderCopy(SDL_Renderer * renderer, SDL_Texture * texture, SDL_Rect * srcrect, SDL_Rect * dstrect)
    cdef int SDL_RenderCopyEx(SDL_Renderer * renderer, SDL_Texture * texture, SDL_Rect * srcrect, SDL_Rect * dstrect, double angle, SDL_Point *center, SDL_RendererFlip flip)
    cdef void SDL_RenderPresent(SDL_Renderer * renderer)
    cdef SDL_bool SDL_RenderTargetSupported(SDL_Renderer *renderer)
    cdef int SDL_SetTargetTexture(SDL_Texture *texture)
    cdef SDL_bool SDL_ResetTargetTexture(SDL_Renderer *renderer)
    cdef void SDL_DestroyTexture(SDL_Texture * texture)
    cdef void SDL_FreeSurface(SDL_Surface * surface)
    cdef int SDL_UpperBlit (SDL_Surface * src, SDL_Rect * srcrect, SDL_Surface * dst, SDL_Rect * dstrect)
    cdef int SDL_LockTexture(SDL_Texture * texture, SDL_Rect * rect, void **pixels, int *pitch)
    cdef void SDL_UnlockTexture(SDL_Texture * texture)
    cdef void SDL_GetWindowSize(SDL_Window * window, int *w, int *h)
    cdef SDL_Window * SDL_CreateWindow(char *title, int x, int y, int w, int h, Uint32 flags)
    cdef int SDL_SetRenderDrawColor(SDL_Renderer * renderer, Uint8 r, Uint8 g, Uint8 b, Uint8 a)
    cdef int SDL_RenderClear(SDL_Renderer * renderer)
    cdef int SDL_SetTextureBlendMode(SDL_Texture * texture, SDL_BlendMode blendMode)
    cdef int SDL_GetTextureBlendMode(SDL_Texture * texture, SDL_BlendMode *blendMode)
    cdef SDL_Surface * SDL_CreateRGBSurfaceFrom(void *pixels, int width, int height, int depth, int pitch, Uint32 Rmask, Uint32 Gmask, Uint32 Bmask, Uint32 Amask)
    cdef int SDL_Init(Uint32 flags)
    cdef void SDL_Quit()
    cdef int SDL_EnableUNICODE(int enable)
    cdef Uint32 SDL_GetTicks()
    cdef void SDL_Delay(Uint32 ms)
    cdef int SDL_PollEvent(SDL_Event * event)
    cdef SDL_RWops * SDL_RWFromFile(char *file, char *mode)
    cdef void SDL_FreeRW(SDL_RWops *area)
    cdef int SDL_GetRendererInfo(SDL_Renderer *renderer, SDL_RendererInfo *info)
    cdef int SDL_RenderSetViewport(SDL_Renderer * renderer, SDL_Rect * rect)
    cdef int SDL_GetCurrentDisplayMode(int displayIndex, SDL_DisplayMode * mode)
    cdef int SDL_GetDesktopDisplayMode(int displayIndex, SDL_DisplayMode * mode)
    cdef int SDL_SetTextureColorMod(SDL_Texture * texture, Uint8 r, Uint8 g, Uint8 b)
    cdef int SDL_SetTextureAlphaMod(SDL_Texture * texture, Uint8 alpha)
    cdef char * SDL_GetError()

cdef extern from "SDL_image.h":
    cdef SDL_Surface *IMG_Load(char *file)

cdef extern from "SDL_ttf.h":
    ctypedef struct TTF_Font
    cdef int TTF_Init()
    cdef TTF_Font *  TTF_OpenFont( char *file, int ptsize)
    cdef TTF_Font *  TTF_OpenFontIndex( char *file, int ptsize, long index)
    cdef TTF_Font *  TTF_OpenFontRW(SDL_RWops *src, int freesrc, int ptsize)
    cdef TTF_Font *  TTF_OpenFontIndexRW(SDL_RWops *src, int freesrc, int ptsize, long index)
    #Set and retrieve the font style
    #define TTF_STYLE_NORMAL    0x00
    #define TTF_STYLE_BOLD      0x01
    #define TTF_STYLE_ITALIC    0x02
    #define TTF_STYLE_UNDERLINE 0x04
    #define TTF_STYLE_STRIKETHROUGH 0x08
    cdef int  TTF_GetFontStyle( TTF_Font *font)
    cdef void  TTF_SetFontStyle(TTF_Font *font, int style)
    cdef int  TTF_GetFontOutline( TTF_Font *font)
    cdef void  TTF_SetFontOutline(TTF_Font *font, int outline)

    #Set and retrieve FreeType hinter settings */
    #define TTF_HINTING_NORMAL    0
    #define TTF_HINTING_LIGHT     1
    #define TTF_HINTING_MONO      2
    #define TTF_HINTING_NONE      3
    cdef int  TTF_GetFontHinting( TTF_Font *font)
    cdef void  TTF_SetFontHinting(TTF_Font *font, int hinting)

    #Get the total height of the font - usually equal to point size
    cdef int  TTF_FontHeight( TTF_Font *font)

    ## Get the offset from the baseline to the top of the font
    #This is a positive value, relative to the baseline.
    #*/
    cdef int  TTF_FontAscent( TTF_Font *font)

    ## Get the offset from the baseline to the bottom of the font
    #   This is a negative value, relative to the baseline.
    # */
    cdef int  TTF_FontDescent( TTF_Font *font)

    ## Get the recommended spacing between lines of text for this font */
    cdef int  TTF_FontLineSkip( TTF_Font *font)

    ## Get/Set whether or not kerning is allowed for this font */
    cdef int  TTF_GetFontKerning( TTF_Font *font)
    cdef void  TTF_SetFontKerning(TTF_Font *font, int allowed)

    ## Get the number of faces of the font */
    cdef long  TTF_FontFaces( TTF_Font *font)

    ## Get the font face attributes, if any */
    cdef int  TTF_FontFaceIsFixedWidth( TTF_Font *font)
    cdef char *  TTF_FontFaceFamilyName( TTF_Font *font)
    cdef char *  TTF_FontFaceStyleName( TTF_Font *font)

    ## Check wether a glyph is provided by the font or not */
    cdef int  TTF_GlyphIsProvided( TTF_Font *font, Uint16 ch)

    ## Get the metrics (dimensions) of a glyph
    #   To understand what these metrics mean, here is a useful link:
    #    http://freetype.sourceforge.net/freetype2/docs/tutorial/step2.html
    # */
    cdef int  TTF_GlyphMetrics(TTF_Font *font, Uint16 ch,int *minx, int *maxx, int *miny, int *maxy, int *advance)

    ## Get the dimensions of a rendered string of text */
    cdef int  TTF_SizeText(TTF_Font *font,  char *text, int *w, int *h)
    cdef int  TTF_SizeUTF8(TTF_Font *font,  char *text, int *w, int *h)
    cdef int  TTF_SizeUNICODE(TTF_Font *font,  Uint16 *text, int *w, int *h)

    # Create an 8-bit palettized surface and render the given text at
    #   fast quality with the given font and color.  The 0 pixel is the
    #   colorkey, giving a transparent background, and the 1 pixel is set
    #   to the text color.
    #   This function returns the new surface, or NULL if there was an error.
    #*/
    cdef SDL_Surface *  TTF_RenderText_Solid(TTF_Font *font, char *text, SDL_Color fg)
    cdef SDL_Surface *  TTF_RenderUTF8_Solid(TTF_Font *font, char *text, SDL_Color fg)
    cdef SDL_Surface *  TTF_RenderUNICODE_Solid(TTF_Font *font, Uint16 *text, SDL_Color fg)

    # Create an 8-bit palettized surface and render the given glyph at
    #   fast quality with the given font and color.  The 0 pixel is the
    #   colorkey, giving a transparent background, and the 1 pixel is set
    #   to the text color.  The glyph is rendered without any padding or
    #   centering in the X direction, and aligned normally in the Y direction.
    #   This function returns the new surface, or NULL if there was an error.
    #*/
    cdef SDL_Surface *  TTF_RenderGlyph_Solid(TTF_Font *font, Uint16 ch, SDL_Color fg)

    # Create an 8-bit palettized surface and render the given text at
    #   high quality with the given font and colors.  The 0 pixel is background,
    #   while other pixels have varying degrees of the foreground color.
    #  This function returns the new surface, or NULL if there was an error.
    #*/
    cdef SDL_Surface *  TTF_RenderText_Shaded(TTF_Font *font, char *text, SDL_Color fg, SDL_Color bg)
    cdef SDL_Surface *  TTF_RenderUTF8_Shaded(TTF_Font *font, char *text, SDL_Color fg, SDL_Color bg)
    cdef SDL_Surface *  TTF_RenderUNICODE_Shaded(TTF_Font *font, Uint16 *text, SDL_Color fg, SDL_Color bg)

    # Create an 8-bit palettized surface and render the given glyph at
    #   high quality with the given font and colors.  The 0 pixel is background,
    #   while other pixels have varying degrees of the foreground color.
    #   The glyph is rendered without any padding or centering in the X
    #   direction, and aligned normally in the Y direction.
    #   This function returns the new surface, or NULL if there was an error.
    #
    cdef SDL_Surface *  TTF_RenderGlyph_Shaded(TTF_Font *font,
                    Uint16 ch, SDL_Color fg, SDL_Color bg)

    # Create a 32-bit ARGB surface and render the given text at high quality,
    #   using alpha blending to dither the font with the given color.
    #   This function returns the new surface, or NULL if there was an error.
    #*/
    cdef SDL_Surface *  TTF_RenderText_Blended(TTF_Font *font,
                     char *text, SDL_Color fg)
    cdef SDL_Surface *  TTF_RenderUTF8_Blended(TTF_Font *font,
                     char *text, SDL_Color fg)
    cdef SDL_Surface *  TTF_RenderUNICODE_Blended(TTF_Font *font,
                     Uint16 *text, SDL_Color fg)

    # Create a 32-bit ARGB surface and render the given glyph at high quality,
    #   using alpha blending to dither the font with the given color.
    #   The glyph is rendered without any padding or centering in the X
    #   direction, and aligned normally in the Y direction.
    #   This function returns the new surface, or NULL if there was an error.
    #*/
    cdef SDL_Surface *  TTF_RenderGlyph_Blended(TTF_Font *font,
                            Uint16 ch, SDL_Color fg)

    # For compatibility with previous versions, here are the old functions */
    #define TTF_RenderText(font, text, fg, bg)  \
    #    TTF_RenderText_Shaded(font, text, fg, bg)
    #define TTF_RenderUTF8(font, text, fg, bg)  \
    #    TTF_RenderUTF8_Shaded(font, text, fg, bg)
    #define TTF_RenderUNICODE(font, text, fg, bg)   \
    #    TTF_RenderUNICODE_Shaded(font, text, fg, bg)

    # Close an opened font file */
    cdef void  TTF_CloseFont(TTF_Font *font)

    # De-initialize the TTF engine */
    cdef void  TTF_Quit()

    # Check if the TTF engine is initialized */
    cdef int  TTF_WasInit()

    # Get the kerning size of two glyphs */
    cdef int TTF_GetFontKerningSize(TTF_Font *font, int prev_index, int index)

 

Programming, Python

No comments

If you are interested in cross compiling Python 2.7.2 for use in Windows using the Mingw32 cross compiling tools I’ll suggest that you first read my post on how to cross compile it for Android as much of the instructions are the same.

Again, this instructions are tested under Ubuntu 11.04 (Natty) 64 bits . The patch that you’ll need is based on the Android patch, and several other pieces from around the web (including several proposed patches in the Python tracker).

Instead of repeating the instructions all over again, I’ll just point out the differences in the process. Assuming you’ve already downloaded Python’s 2.7.2 source, the patch, created a host version of the Python interpreter and applied the patch for cross compilation, you’ll then need to install the Mingw32 toolset (if you haven’t already):

sudo apt-get install mingw32 mingw32-binutils mingw32-runtime

Prepare the environment:

export ARCH="win32"
export CFLAGS=""
export CXXFLAGS=""
export CC="i586-mingw32msvc-gcc"
export CXX="i586-mingw32msvc-g++"
export AR="i586-mingw32msvc-ar"
export RANLIB="i586-mingw32msvc-ranlib"
export STRIP="i586-mingw32msvc-strip --strip-unneeded"
export LD="i586-mingw32msvc-ld"
export AS="i586-mingw32msvc-as"
export NM="i586-mingw32msvc-nm"
export DLLTOOL="i586-mingw32msvc-dlltool"
export OBJDUMP="i586-mingw32msvc-objdump"
export RESCOMP="i586-mingw32msvc-windres"
export MAKE="make -k -j4 HOSTPYTHON=[PATH TO HOST PYTHON] HOSTPGEN=[PATH TO HOST PGEN]  CROSS_COMPILE=mingw32msvc CROSS_COMPILE_TARGET=yes"
export EXTRALIBS="-lstdc++ -lgcc -lodbc32 -lwsock32 -lwinspool -lwinmm -lshell32 -lcomctl32 -lctl3d32 -lodbc32 -ladvapi32 -lopengl32 -lglu32 -lole32 -loleaut32 -luuid"

Then configure the build:

./configure LDFLAGS="-Wl,--no-export-dynamic -static-libgcc -static $EXTRALIBS" CFLAGS="-DMS_WIN32 -DMS_WINDOWS -DHAVE_USABLE_WCHAR_T" CPPFLAGS="-static" LINKFORSHARED=" " LIBOBJS="import_nt.o dl_nt.o getpathp.o" THREADOBJ="Python/thread.o" DYNLOADFILE="dynload_win.o" --disable-shared HOSTPYTHON=[PATH TO HOST PYTHON] HOSTPGEN=[PATH TO HOST PGEN] --host=i586-mingw32msvc --build=i686-pc-linux-gnu  --prefix="[WHERE YOU WANT TO PUT THE GENERATED PYTHON STUFF]"

You may have noticed that there’s a few static compilation related switches in there. I think it’s possible to compile Python without them, but as I’m building a monolithic app with all the modules compiled in the same exe file, I haven’t really tested the shared library building part of the process. If you do test with shared libraries enabled, then feel free to leave feedback in the comments and I’ll update the article with whatever caveats there might be.

Now comes the hacky stuff…not everything that should be fixed for the cross compilation to succeed is fixed in the patch, so we need to do some manual repairs after the config process is finished.

sed -i "s|\${LIBOBJDIR}fileblocks\$U\.o||g" Makefile
# Enable NT Threads
sed -i "s|.*NT_THREADS.*|#define NT_THREADS|g" pyconfig.h
# Disable PTY stuff that gets activated because of errors in the configure script
sed -i "s|.*HAVE_OPENPTY.*|#undef HAVE_OPENPTY|g" pyconfig.h
sed -i "s|.*HAVE__GETPTY.*|#undef HAVE__GETPTY|g" pyconfig.h
sed -i "s|.*HAVE_DEV_PTMX.*|#undef HAVE_DEV_PTMX|g" pyconfig.h

Finally:

$MAKE

Hopefully you’ll get a static python binary (.exe) and also a static library that you can link against other files. The interactive console seems to work as well (if you copy the generated files to Windows and run python.exe)