Time to turn it off, and say goodbye to Wordpress in one go! Wordpress had a lot of security findings in the past, and I used it as a file hoster anyways.

In this project, I'll take you through my journey of designing and building a custom MIDI DJ controller from scratch. Combining CAD software, 3D printing, and custom microcontroller software, I'll show you how to create a fully functional device that seamlessly integrates both hardware and software—capable of controlling popular DJ software like Mixxx. This step-by-step guide offers a hands-on approach to constructing a personalized MIDI controller, making it perfect for anyone with an interest in electronics, 3D printing, or music technology.

Disclaimer: Prior to this project I've never used any DJ controller!

Layout​

Woah a lot of buttons, knobs and sliders... This is the UI of Mixxx, the virtual counterpart of the planned MIDI controller. As a first step, all the controls that I planned to use, needed to be selected and suitable parts ordered. Let's focus on one side of the UI, as it is mirrored anyways. So I ended up with the following selection of UI elements that I wanted to control:

• Jog wheel - to seek forwards and backwards in tracks
• Pause play tracks
• Sync lock button to sync the speed of tracks
• Temporary slow down and speed up
• Slow down and speed up
• Speed slider
• High, mid, low filter, including their mute buttons
• Filter knobs and their activation buttons
• Volume slider and gain knobs
• Cross-fade slider
• Headphones button to select which side is playing on headphones

Here is the planned layout in CAD compared to Mixxx's UI, side by side:

I ended up with the following components:

• 26 push buttons
• 16 rotary potentiometers
• 5 linear potentiometers
• 2 rotary encoders including push button functionality (in red)
• 12 LEDs

Efficient microcontroller wiring using multiplexers​

Directly wiring all the components would have required a microcontroller with an impractical number of pins—around 65, not even counting ground and power connections. Fortunately, there's a simple solution: multiplexers. I used four 16-channel CD74HC4067 multiplexers to reduce the number of pins needed to interface with multiple components.

How multiplexers work​

A multiplexer acts like a digital switchboard, allowing you to connect multiple signals to a single communication line. With these multiplexers, it is possible to connect up to 64 different signals (4 multiplexers × 16 channels) while only using 6 pins on the microcontroller.

Pin connection breakdown​

• Enable (EN) Pins: These control which multiplexer is active
• Select Pins (S0-S4): Used to choose a specific channel (0-15)
• Signal (SIG) Pin: The communication line for reading or writing

Reading channels: step by step​

Here's the systematic process for reading all channels:

1. Enable a specific multiplexer by activating its EN pin
2. Use the S0-S4 pins to select a specific channel (0-15)
3. Read the signal on the SIG pin
4. Repeat for all 16 channels
5. Move to the next multiplexer and repeat

Writing channels: LEDs​

This setup isn't just for reading signals. It works beautifully for:

• Reading potentiometer values
• Reading button states
• Controlling LEDs

By rapidly cycling through channels, you can control LEDs without noticeable flickering. The speed of modern microcontrollers and the persistence of vision makes this possible.

Additional component connections​

Not everything needs a multiplexer. For instance, the two rotary encoders connect directly to the microcontroller, requiring only three pins each:

• Clock (CLK)
• Data Transfer (DT)
• Switch (SW)

Wiring​

My wiring still got slightly out of hand, but the four multiplexers significantly reduced the complexity and pin requirements, making this project a lot more accessible. When designing complex electronic systems, always look for ways to reduce direct pin connections. Multiplexers, shift registers, and other expansion techniques can be game-changers.

Anyways here's a picture of the wiring. To simplify the microcontroller connection, a green prototyping PCB connects all the signal (SIG), enable (EN), and channel (C0-C15) pins of the four blue multiplexer boards, effectively creating a centralized wiring hub.

Microcontroller​

To power this project, I selected the ESP32-S2 microcontroller—specifically the T8-ESP32-S2-WOOR v1.1—which brings several capabilities to our design. This microcontroller is particularly well-suited for this MIDI controller project due to the following features:

• Native USB functionality, allowing it to act as a MIDI class USB device
• Integrated WiFi for network connectivity
• Sufficient processor power to handle multiple tasks
• Flexible configuration options through onboard dip switches

The dip switches provide a crucial flexibility for device mode selection. They allow toggling between two USB modes:

1. USB mode: Debugging and development
2. OTG (On-The-Go) mode: In our specific implementation, this enables MIDI device functionality

This microcontroller's ability to rapidly iterate through connected components while simultaneously performing signal filtering makes it an excellent choice for complex input mapping and real-time music interface applications.

The microcontroller's dip switches create a programming limitation: USB mode is required for firmware uploads, while OTG mode enables MIDI device functionality. Once the microcontroller is built into the case, the dip switches aren't easily accessible anymore. However, this isn't a significant issue. The initial firmware can be loaded via USB, and subsequent updates can be performed wirelessly using the built-in WiFi through Over-The-Air (OTA) updates. This means you'll only need a physical USB connection once to set up the device, with all future firmware updates happening seamlessly over the network.

Software​

Here's the repository with the code: https://github.com/AJ92/midi-slayer-one

Building the firmware should be done with the Arduino IDE in version 2.x, and ESP32 board manager in version 2.0.4. Version 3.x and newer do not work! Same goes for EspTinyUSB dependency. It has to be the one mentioned in the readme.

The microcontroller implements button debouncing and potentiometer value filtering to reduce analog signal noise and jitter. These signal processing techniques smooth out input signals but can introduce slight input lag. The filtering parameters are configurable, allowing users to balance signal stability and responsiveness according to their specific needs. By mitigating noise from analog inputs and multiplexing, the controller ensures more reliable and consistent performance.

OTA firmware updates are enabled by holding button B0 for 5 seconds after connecting power via USB. All LEDs light up and the ESP32 connects to the hardcoded WiFi access point. The firmware can be uploaded via any browser. After the update the ESP32 has to be restarted. Disconnect and connect it again.

Calibrating potentiometer center and end points is done by holding the button B12 for 5 seconds after connecting power via USB. All LEDs start to blink. Move all potentiometers to their minimum, then to their maximum, and center them. Once all of them are centered, press B12 again, and the calibration is stored.

After wiring make sure to use the correct PINs. Adjust this file to your PIN layout!

For Mixxx you can import the controller mapping midi-slayer-one.midi.xml linked below.

Pictures​

Downloads​

• midi-slayer-one.midi.xml.zip
• cad-midictrl.zip

I had the idea to simulate a Betaflight flight controller for some time and, in 2023, I began porting the code to C#. However, I soon discovered that Betaflight already had a target that does exactly this! The SITL (Software in the Loop) target can be built on a Linux machine and even runs smoothly in WSL (Windows Subsystem for Linux), which is fantastic. However, this setup presents several challenges for the average Windows user.

First, WSL as a dependency complicates the setup. Installing WSL can be a significant hurdle for inexperienced users, and to make matters worse, the host IP address tends to change after each restart. While this issue can be addressed with scripts and IP queries, I wanted to avoid that level of complexity. Diagnosing such problems remotely, especially for non-tech-savvy users, can be difficult, and it also introduces platform dependency. Don’t believe me? Try setting it up yourself using the official guide: Betaflight SITL Guide.

The second major problem is the ping-pong like synchronous communication over the network stack between the virtual flight controller (SITL) and the simulation. The SITL process waits for updated simulation data (fdmPkt) and then generates motor outputs (pwmPkt) that are sent back to the simulation. The simulation interprets these motor outputs and adjusts the virtual quadcopter's orientation accordingly. This, in turn, creates new gyro input for the SITL, and the loop repeats. However, this introduces latency and undesirable artifacts in the simulation, such as sudden jumps in RC data, gyro input, or motor outputs. These issues arise when the simulation doesn't run a perfectly timed, high-frequency update loop, which is virtually impossible. You can check out the code here. It's extremely simple: it receives inputs and responds with outputs. While this simplicity likely aims to offer flexibility, it also limits how fast updates can be processed. Given that an average quadcopter processes gyro samples at 4kHz, you only have 0.25 ms to run the simulation, including the overhead caused by UDP-based network communication, threading, and the OS scheduler!

And lastly, where are the OSD and blackbox features? Unfortunately, SITL doesn’t include these out of the box.

Fortunately, KwadSim and its counterpart, KwadSimITL, were mentioned on the Betaflight Discord, and they address some of the shortcomings of the Betaflight SITL target. First off, KwadSim comes with a Windows CMake toolchain that uses MinGW, eliminating the need for WSL. That's a big win—no more WSL dependency!

KwadSimITL also features parameterized physics and allows to run multiple simulation steps with a single input. In other words, a single input packet can generate a finer simulation resolution. This shifts some of the computational load from the game client to the SITL process, as a low-frequency game physics loop can trigger multiple SITL physics updates. For example, the game client's physics loop might run at 60Hz, but each update could trigger 100 SITL physics updates, providing the PID loop with 6kHz of fresh, unique gyro inputs.

But that’s not all—OSD and blackbox recording are already integrated and fully functional.

However, KwadSimITL isn't without its flaws. The project hasn't been updated since early 2020 and is based on Betaflight version 4.1.0, which is quite outdated. But the most significant challenge for me was the game client. KwadSim uses Godot 3.2, a game engine I have no experience with. So, I decided to rewrite the game client from scratch in Unity—and that’s how pr0p was born.

This is my attempt at creating a quad copter simulation based on the previous work of KwadSimITL and a few of the biggest problems I tried to solve.

fun with state updates​

Updating the simulation state across two processes was a tough challenge, particularly because any added latency in this step was immediately noticeable in the overall flight experience. Since the default SITL target simply responds to requests from the simulation process, it's important to understand the relationship between these two processes.

It's worth noting that while the functions used to demonstrate this timing problem may not take long in practice, they help illustrate the issue. Let’s assume the simulation runs at 60Hz, and the "simulate" function of the simulation process generates a new simulation state (fdmPkt) for the SITL process. The SITL process then receives this data, generates motor outputs (pwmPkt), and sends them back to the simulation process. Once the motor outputs are received, the simulation updates its state in the next step, feeding the new information back to the SITL process, and the loop continues.

If the goal is to reduce latency by pushing the update frequency to its limits, the process would look something like this:

The gap between function pairs narrows, and simulations are called more frequently than before. However, the update frequency can only be pushed so far—if it goes too high, the simulation may miss motor inputs. If the simulation runs ahead without waiting for fresh motor outputs, it ends up calculating two packets based on the same motor data.

Why is this behavior undesirable?

In simple terms, the simulation requests new motor output values from the SITL process. SITL’s PID loop calculates these motor outputs, and the simulation uses them to update the quadcopter's orientation. If the simulation step begins before receiving fresh motor outputs, it calculates the orientation based on outdated data. In the worst case, the new orientation will be the same as the previous one, or it may miss a critical change that would have occurred with the updated motor outputs. When this stale orientation data is fed back into the PID loop, the motor outputs may increase because no orientation change was detected. It’s as if an external force is preventing the quadcopter from responding to control inputs for an additional simulation step.

This effect is something most people have experienced in real life: if a quadcopter’s motors are spinning without propellers and it’s armed with RC inputs requesting orientation changes, the motors will keep increasing RPMs. This happens because the flight controller detects no orientation change, even though the motors are spinning faster and faster, so it continues to push harder.

To summarize, SITL needs input that reflects the previous output—meaning the orientation must respond to motor outputs with every round trip. If this isn’t guaranteed, motor outputs may become higher than expected, either for single or multiple update steps.

KwadSim(ITL) has a nice concept for this problem. It introduced 2 simulations, one for each process. The SITL process has it's own physics simulation, the same that the game client has and both run more or less independent fom each other, but include updates from each other. The game client receives the orientation and acceleration updates, calculated from SITL physics updates, and the game client sends its version of these values back.

This concept is modified slightly for pr0p and its SITL fork, SimITL. The game client (pr0p) handles the complex task of calculating collisions between the quadcopter and the environment, which can be CPU-intensive. SimITL, on the other hand, focuses solely on quadcopter physics—handling the motors, propellers, frame, and overall behavior as if the quadcopter were flying in free space without obstacles or collisions. This makes SimITL’s task simpler and less CPU-intensive per update step, allowing it to run at a much higher frequency—currently around 3.2kHz—to generate realistic motor noise. The game client, meanwhile, skips these detailed flight dynamics calculations and simply incorporates the acceleration updates from the SimITL process whenever they are received.

This diagram illustrates pr0p’s simulation updates, which are executed almost simultaneously at a single time point (7 state packets in the first line) due to Unity's physics simulation running in a simple loop with constant time deltas. Thanks to the rapid response from SITL (first packet in the second line), pr0p is able to incorporate this new data into one of its subsequent state updates. The time scale is in microseconds, and the events were recorded using Wireshark. Notably, the SITL process responds incredibly quickly to pr0p's initial state update—within approximately 40 microseconds in this specific time window. This near-instant response is achieved by using separate threads for networking on both sides, which actively spin to immediately respond to incoming data.

The beauty of this approach lies in its speed, even when operating over the network stack. Both processes integrate new packets immediately upon receipt, while SITL fills in the gaps by calculating finer steps between packets to generate more realistic motor and propeller noise. In the event of a collision, pr0p uses its own physics engine to handle the majority of the calculations, blending them with SITL's orientation data. This combined result is then fed back to SITL, allowing the flight controller to respond accurately to collisions.

Of course, this entire process is a simplified depiction. Input queues can discard packets if too many are received before processing, which may lead to sudden jumps in orientation and cause the PID loop to react harshly. Additionally, the operating system's scheduler can delay the execution of a process, potentially preventing an immediate response from SITL. Despite these challenges, the system performs well overall, delivering a convincing flight experience with impressive responsiveness and reasonable CPU utilization.

Update:

I received a suggestion to simplify and speed up the process by building SITL as a shared library and calling the simulation functions directly from the fixed game loop. After adjusting the interface and tinkering with compiler settings, I now have a native SITL library that performs faster than all previously mentioned methods. The result? Noticeably reduced latency and lower CPU load. Gone are the unnecessary spinning threads waiting for packets—now it's just efficient, direct function calls.

You can try it yourself—available since version 0.8.x pr0p.dev/download

After another bad crash in a tall tree and a recovery a few weeks later, the copter was in a bad shape and I decided to improve the disassemble/assemble process by using a single TPU canopy, instead of 3 carbon fiber plates ( 2 side plates + top plate + 4 standoffs ).

Design​

Now that the copter is discontinued and the replacement parts will be unavailable soon, I decided to design my own 3" frame, which should be extremely durable, a bit more roomy inside, and easy to assemble / swap out broken parts. I started by scanning the remaining XJB 145 frame plate I had, so the mounting holes for the motors and stack would be perfect. The frame was streched a bit to have more room inside and fit a bigger camera, which was a problem bofore and lead to the tall TPU canopy to clear the propellers. In addition to that, the most important update was durability. I wanted to get away from the 3mm thick bottom carbon fiber plate and use 5mm thick independant arms, so the arm mounting concept of the five33 switchback pro frame got adopted.

Each arm is held in place by one single screw, which goes through a bottom plate, the arm, a top plate and screws into a pressnut on the upper plate and into a standoff. Those screws are m3 now, to withstand crashes better and a single screw is enough here, as the arms fit like a puzzle into the center wiggle plate, and can not rotate anymore. A lower and upper carbon fiber plate is pressing them thogether and the arms are sitting perfectly tight in frame. The 2mm thick center wiggle plate has 4 screws for the stack and at the same time holds itself to the upper plate of the arm sandwich. The 3mm gap between the wiggle plate and the lower plate is reserved for a lipo strap.

The bottom plate has a front grill to provide a little bit of impact protection for the camera and holes and space to mount additional hardware like a buzzer or a RX. The upper plate has also a small grill which can be used to ziptie a capacitor if needed.

Two spikes for the arms should protect the motors from impacts. Also the arms are now a bit thinner and have a cross section of 6mm x 5mm. This reduces blocking of the airflow, but still provides enough strength to survive a few crashes.

A top plate, camera and antenna mount complete the vtx-slayer-one. The mounts are 3d printed with TPU and are flexible enough to survive crashes without breaking immediately and still protect the camera and antenna. The top plate has holes to mount additional hardware like a camera.

First prototype​

The final cad design was sent to a fpv buddy with a cnc milling machine for final checks of holes and tolerances. ~43 grams should be the total weight of the frame with m3 screws and aluminum standoffs.

By adding new spare motors, a runcam phoenix 2 camera, the original hglrc stack ( esc + fc ), a new elrs happymodel EP2 RX and a new EWRF E7082VM VTX, the build was complete. Total weight with a 850mAh 4s lipo is ~280g. On the software side the rx received a fw update, and the vtx was flashed with openVTX. After fixing motor directions and configuring betaflight the 3" vtx-slayer-one was ready for its first flight.

Assembly​

Parts:

• 1x topplate 1,5mm
• 4x arms 5mm
• 1x top sandwich plate 2mm (for arm and stack mount)
• 1x bottom sandwich plate 2mm (for arm mount and stack mount)
• 1x armconnector 2mm (center wiggle plate for arm mount)
• 5x M3x8mm
• 5x M3x16mm
• 5x pressnuts
• 16x M2x7mm motorscrews
• 4x M2x20mm stackscrews
• 8x M2 (nylock)nuts for stack screws
• 1x TPU cam mount
• 1x TPU dipole antenna mount
• 4x M3x20mm standoffs (15mm for a slammed build)

Cost ~46€

The 4" variant on the race track​

The 4" frame with 1407 3700KV (4s) motors performed decent during DCS Aichtal. Rear motors became hot, but that's the only problem i experienced. Arms and the frame in general experienced no noticable damage, even after a few midairs. Over 55 packs where flown during the event, all on one quadcopter.

After an upgrade to 1606 3300KV (4s) motors over 36 packs where emtied with the same quadcopter that I used during DCS and the same overall reliability could be observer. No broken motors, no broken arms. I also used the same old props.

Update 1.3​

• increased arm strength around the motor holes

• added 25.5 x 25.5 mm stack/AIO mounting holes

• added 20 x 20 mm mounting holes to the top plate (e.g. vista mount)

• added new vtx antenna mount

• VTX-SLAYER-1.3.zip

Update 1.4​

• 4" arms

• caddx vista antenna mount

• VTX-SLAYER-1.4.zip

Update 2.2​

• TPU canopy

• amost-true-x arm layout

• removed press-nuts, thick enough standoffs are enough

• parametric arm length

• VTX-SLAYER-2.2.zip

Update 2.3​

• added 5cm LED strip insert in canopy fin

• added strength to fin base

• changed canopy air holes

• stronger arms around m3 holes

• VTX-SLAYER-2.3

Split canopy​

Cam part is split from rest of the canopy. This allows the fin to be printed upright, and has the benefit of easier access to the electronics, without removing the cam all the time.

• slayer-4-v2.3-split-canopy

Happy pilots :D​

Two steps are needed around the script. The server service has to be stopped first, and it has to be started again after the script finished. The copy command writes into ts3/, which is the folder that holds the actual server files. Adjust that path for your layout.

#!/bin/sh
if [ $# -eq 0 ]
  then
    echo "Add url parameter to download"
    exit
fi
mkdir ts3_dl
wget $1 -P ts3_dl
tar -xjf ts3_dl/teamspeak3*.tar.bz2 -C ts3_dl
cp -r ts3_dl/teamspeak3*/* ts3/
rm -r ts3_dl

Run it like this, and give the archive url as the only argument:

./ts3-update.sh <archive-url>

First change to the dir of the log file.

cd /var/log/apache2

This line uses all access.log files, also those of previous days:

zcat access.log.*.gz | goaccess access.log --log-format=COMMON

This line uses all other_vhosts_access.log files, also those previous days:

zcat other_vhosts_access.log.*.gz | goaccess other_vhosts_access.log --log-format=VCOMMON

It should look something like this:

What is it about?​

As I am currently the only developer working on this game in my spare time, I want to keep the game fairly simple. It is a turn based online multiplayer strategy game with two sides, blue and red, that fight over resources, construct fortifications and try to eliminate each other.

What is already done?​

Let's start with the server side. The server is powered by https://nodejs.org/ and uses https://colyseus.io/ to communicate with https://unity.com clients. Why nodejs and colyseus? It is cross-platform, runs fairly well on my vps, as well as local machine and is easy to deploy. And the most benefit is, no need to run unity on the server side.

Following components of the server are already more or less done:

• user registrations
• user/client authentications
• presistent data storage (pgsql)
• game room creation
• random map generation
• unit creation
• unit movement
• unit attacks
• turns
• web based chat

Why unity on the client side? This has a bunch of reasons, but just to name a few: unity is "free", has a ton of tutorials and a fairly nice documentation. Yes I point at you unreal! Get your docs together :). Did I mention it is cross platform? I plan to release the game as a mobile (android) version first.

The client side has already the following features:

• login screen
• a simple main menu
• join a game room
• create a new game room
• rejoin same game room on crash / lost connection
• visualize the server's map
• camera movement / zoom
• basic unit movement
• basic unit attacks
• visualization of the unit's possible moves/attacks
• basic UI
• ingame chat

What is planned ?​

There are quite a few features that I want to add, but at the moment I try to keep the feature set quite limited. Otherwise I fear that nothing will be finished in my life time...

The main goal is to get a simple online multiplayer game running, so at the beginning I will try to restrict myself to only implement the absolute basics like some sort of player level progression with basic statistics about the player. Something like the count of played, won and lost matches. Also the game mechanics will stay quite simple. There will be basic attacks for each unit type. The only differences will be how far and how strong attacks are based on the unit type and the current terrain the unit is positioned at. Looking at the first "playable" version, only a few components are missing. I need some sort of simple Ai, so users don't have to wait until someone is available to play against. And the game logic needs winning conditions, fancy animations and sounds.

I guess that's enough for a first post. I will try to post a bit more frequently about this project :)

This is compressor: http://compressor.sigsegowl.xyz (discontinued)

It only works more or less for static sites. Everything that requires a login on dynamic elements, forms, javascript or all the other fancy things won't work correctly.

This is how a techcrunch article looks like.

This is an arstechnica article.

Images are currently also replaced by a placeholder which links to the original just in case.

All links are replaced by a compressor link so following links should lead to the same stripping like opening an url.

If you like it, use it, if you abuse it, it will not stay public anymore! Source code will be released soon on github.

The idea in June 2014​

My laptop GPU died, and the engine work had to wait. So I started a small tool instead. It reads the sensor information of a desktop machine and sends it to an Android device for monitoring. The server reads the sensor data from HWInfo, and the server side was almost done within weeks.

The plan for the code was this: the sensor retrieval server stays free and closed source, at least the sensor retrieval code and the shared memory layout. The Android client gets published without charge and might be fully open source.

Features in August 2014​

• monitor all sensors that HWInfo offers
• work inside a WLAN or LAN network, or over the internet
• a SQLite database on the Android side to keep the settings
• save multiple servers and their sensors
• planned: gauges, bars, more options

Alpha in February 2015​

The app reached alpha status and the UI was still rough. Three files are needed to run the setup, because the client alone does nothing.

• HWInfo: get it from hwinfo.com
• desktop server: HwInfoTransmitter-release-v1.0.zip
• Android client: HwInfoReceiver-debug.zip

The important step is inside HWInfo itself. The shared memory option must be enabled, and the sensors must be running. Without that step the server has no data to read.

New GUI in October 2016​

The old layout showed few gauges on the screen. The new GUI fits many more gauges, and it was still work in progress. The remaining steps were the font size, the connection of gauges to sensor data, the setup UI for gauges and a layout bug with scrolling.

In the same month the app got the short name HwR, and the open beta went live on the Playstore. The beta allowed 1000 downloads, because the version could be very unstable.

• HwR on the Playstore
• desktop build for that beta: HwInfoTransmitterReleaseOpenBeta.zip

Update 1.5 in February 2017​

New in the app:

• keep the screen on while you view sensors, as a setting
• add sensors by name from the phone instead of by id
• bug fixes

The desktop counterpart got the same update. Get the desktop build here. Two days later I fixed the desktop app, because it did not respond to client requests.

Update 1.6 in May 2018​

• fixed font colors for the dark theme
• custom scale for the texts next to the gauges
• custom scale for temperature values
• minimum and maximum values now scale automatically depending on the input values

This is the state of the app today. The Playstore link above is the current download.

Ever tried to render something with openGL on a linux server that has no x11 installed nor has a physical gpu connected? I searched for a long time for a solution that would work almost out of the box and the first solution i could find was actually installing x11, mesa3d and a dummy x-display. Via putty that has x11-forwarding enabled it is possible to run x-applications just fine and even render with openGL thanks to mesa in software mode!

But as soon as you close the ssh connection x11 stops and after that x-applications and openGL won't be able to connect to a display! So after every server restart openGL won't work anymore...

So why do you need a display or x11 if you want to render offscreen?

The answer is you don't! Here is a quick test:

1. Install libraries​

I set up a virtual machine in virtual box based on ubuntu server 17.04 and installed the following libs to get started:

• mesa for openGL software rendering
• osmesa for offscreen rendering
• glu1-mesa for some glu :)
• gcc to compile c

sudo apt install mesa-common-dev
sudo apt install libosmesa6-dev
sudo apt install libglu1-mesa-dev
sudo apt install gcc

2. Create source file​

Now lets create the osmesa demo which i have modified slightly to compile out of the box:

sudo nano osdemo.c

Paste the following code in nano:

/*
based on: 
 
https://github.com/JoakimSoderberg/mesademos/blob/master/src/osdemos/osdemo32.c
*/
 
#include <stdio.h>
#include <stdlib.h>
#include <GL/osmesa.h>
#include <GL/glu.h>     //replaced by default header in linux
 
#define SAVE_TARGA
 
#define WIDTH 400
#define HEIGHT 400
 
static void render_image( void )
{
   GLfloat light_ambient[] = { 0.0, 0.0, 0.0, 1.0 };
   GLfloat light_diffuse[] = { 1.0, 1.0, 1.0, 1.0 };
   GLfloat light_specular[] = { 1.0, 1.0, 1.0, 1.0 };
   GLfloat light_position[] = { 1.0, 1.0, 1.0, 0.0 };
   GLfloat red_mat[]   = { 1.0, 0.2, 0.2, 1.0 };
   GLfloat green_mat[] = { 0.2, 1.0, 0.2, 0.5 };
   GLfloat blue_mat[]  = { 0.2, 0.2, 1.0, 1.0 };
   GLfloat white_mat[]  = { 1.0, 1.0, 1.0, 1.0 };
   GLfloat purple_mat[] = { 1.0, 0.2, 1.0, 1.0 };
   GLUquadricObj *qobj = gluNewQuadric();
 
   glLightfv(GL_LIGHT0, GL_AMBIENT, light_ambient);
   glLightfv(GL_LIGHT0, GL_DIFFUSE, light_diffuse);
   glLightfv(GL_LIGHT0, GL_SPECULAR, light_specular);
   glLightfv(GL_LIGHT0, GL_POSITION, light_position);
 
   glEnable(GL_LIGHTING);
   glEnable(GL_LIGHT0);
   glEnable(GL_DEPTH_TEST);
 
   glMatrixMode(GL_PROJECTION);
   glLoadIdentity();
   glOrtho(-2.5, 2.5, -2.5, 2.5, -10.0, 10.0);
   glMatrixMode(GL_MODELVIEW);
 
   glClear( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT );
 
   glPushMatrix();
   glRotatef(20.0, 1.0, 0.0, 0.0);
 
#if 0
   glPushMatrix();
   glTranslatef(-0.75, 0.5, 0.0); 
   glRotatef(90.0, 1.0, 0.0, 0.0);
   glMaterialfv( GL_FRONT_AND_BACK, GL_AMBIENT_AND_DIFFUSE, red_mat );
   glutSolidTorus(0.275, 0.85, 20, 20);
   glPopMatrix();
#endif
 
   /* red square */
   glPushMatrix();
   glTranslatef(0.0, -0.5, 0.0); 
   glRotatef(90, 1, 0.5, 0);
   glScalef(3, 3, 3);
   glDisable(GL_LIGHTING);
   glColor4f(1, 0, 0, 0.5);
   glBegin(GL_POLYGON);
   glVertex2f(-1, -1);
   glVertex2f( 1, -1);
   glVertex2f( 1,  1);
   glVertex2f(-1,  1);
   glEnd();
   glEnable(GL_LIGHTING);
   glPopMatrix();
 
#if 0
   /* green square */
   glPushMatrix();
   glTranslatef(0.0, 0.5, 0.1); 
   glDisable(GL_LIGHTING);
   glColor3f(0, 1, 0);
   glBegin(GL_POLYGON);
   glVertex2f(-1, -1);
   glVertex2f( 1, -1);
   glVertex2f( 1,  1);
   glVertex2f(-1,  1);
   glEnd();
   glEnable(GL_LIGHTING);
   glPopMatrix();
   /* blue square */
   glPushMatrix();
   glTranslatef(0.75, 0.5, 0.3); 
   glDisable(GL_LIGHTING);
   glColor3f(0, 0, 0.5);
   glBegin(GL_POLYGON);
   glVertex2f(-1, -1);
   glVertex2f( 1, -1);
   glVertex2f( 1,  1);
   glVertex2f(-1,  1);
   glEnd();
   glEnable(GL_LIGHTING);
   glPopMatrix();
#endif
   glPushMatrix();
   glTranslatef(-0.75, -0.5, 0.0); 
   glRotatef(270.0, 1.0, 0.0, 0.0);
   glMaterialfv( GL_FRONT_AND_BACK, GL_AMBIENT_AND_DIFFUSE, green_mat );
   glColor4f(0,1,0,0.5);
   glEnable(GL_BLEND);
   glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
   gluCylinder(qobj, 1.0, 0.0, 2.0, 16, 1);
   glDisable(GL_BLEND);
   glPopMatrix();
 
   glPushMatrix();
   glTranslatef(0.75, 1.0, 1.0); 
   glMaterialfv( GL_FRONT_AND_BACK, GL_AMBIENT_AND_DIFFUSE, blue_mat );
   gluSphere(qobj, 1.0, 20, 20);
   glPopMatrix();
 
   glPopMatrix();
 
   /* This is very important!!!
    * Make sure buffered commands are finished!!!
    */
   glFinish();
 
   gluDeleteQuadric(qobj);
 
   {
      GLint r, g, b, a;
      glGetIntegerv(GL_RED_BITS, &r);
      glGetIntegerv(GL_GREEN_BITS, &g);
      glGetIntegerv(GL_BLUE_BITS, &b);
      glGetIntegerv(GL_ALPHA_BITS, &a);
      printf("channel sizes: %d %d %d %d\n", r, g, b, a);
   }
}
 
 
 
static void
write_targa(const char *filename, const GLfloat *buffer, int width, int height)
{
   FILE *f = fopen( filename, "w" );
   if (f) {
      int i, x, y;
      const GLfloat *ptr = buffer;
      printf ("osdemo, writing tga file \n");
      fputc (0x00, f);	/* ID Length, 0 => No ID	*/
      fputc (0x00, f);	/* Color Map Type, 0 => No color map included	*/
      fputc (0x02, f);	/* Image Type, 2 => Uncompressed, True-color Image */
      fputc (0x00, f);	/* Next five bytes are about the color map entries */
      fputc (0x00, f);	/* 2 bytes Index, 2 bytes length, 1 byte size */
      fputc (0x00, f);
      fputc (0x00, f);
      fputc (0x00, f);
      fputc (0x00, f);	/* X-origin of Image	*/
      fputc (0x00, f);
      fputc (0x00, f);	/* Y-origin of Image	*/
      fputc (0x00, f);
      fputc (WIDTH & 0xff, f);      /* Image Width	*/
      fputc ((WIDTH>>8) & 0xff, f);
      fputc (HEIGHT & 0xff, f);     /* Image Height	*/
      fputc ((HEIGHT>>8) & 0xff, f);
      fputc (0x18, f);		/* Pixel Depth, 0x18 => 24 Bits	*/
      fputc (0x20, f);		/* Image Descriptor	*/
      fclose(f);
      f = fopen( filename, "ab" );  /* reopen in binary append mode */
      for (y=height-1; y>=0; y--) {
         for (x=0; x<width; x++) { int r, g, b; i = (y*width + x) * 4; r = (int) (ptr[i+0] * 255.0); g = (int) (ptr[i+1] * 255.0); b = (int) (ptr[i+2] * 255.0); if (r > 255) r = 255;
            if (g > 255) g = 255;
            if (b > 255) b = 255;
            fputc(b, f); /* write blue */
            fputc(g, f); /* write green */
            fputc(r, f); /* write red */
         }
      }
   }
}
 
 
static void
write_ppm(const char *filename, const GLfloat *buffer, int width, int height)
{
   const int binary = 0;
   FILE *f = fopen( filename, "w" );
   if (f) {
      int i, x, y;
      const GLfloat *ptr = buffer;
      if (binary) {
         fprintf(f,"P6\n");
         fprintf(f,"# ppm-file created by osdemo.c\n");
         fprintf(f,"%i %i\n", width,height);
         fprintf(f,"255\n");
         fclose(f);
         f = fopen( filename, "ab" );  /* reopen in binary append mode */
         for (y=height-1; y>=0; y--) {
            for (x=0; x<width; x++) { int r, g, b; i = (y*width + x) * 4; r = (int) (ptr[i+0] * 255.0); g = (int) (ptr[i+1] * 255.0); b = (int) (ptr[i+2] * 255.0); if (r > 255) r = 255;
               if (g > 255) g = 255;
               if (b > 255) b = 255;
               fputc(r, f);   /* write red */
               fputc(g, f); /* write green */
               fputc(b, f); /* write blue */
            }
         }
      }
      else {
         /*ASCII*/
         int counter = 0;
         fprintf(f,"P3\n");
         fprintf(f,"# ascii ppm file created by osdemo.c\n");
         fprintf(f,"%i %i\n", width, height);
         fprintf(f,"255\n");
         for (y=height-1; y>=0; y--) {
            for (x=0; x<width; x++) { int r, g, b; i = (y*width + x) * 4; r = (int) (ptr[i+0] * 255.0); g = (int) (ptr[i+1] * 255.0); b = (int) (ptr[i+2] * 255.0); if (r > 255) r = 255;
               if (g > 255) g = 255;
               if (b > 255) b = 255;
               fprintf(f, " %3d %3d %3d", r, g, b);
               counter++;
               if (counter % 5 == 0)
                  fprintf(f, "\n");
            }
         }
      }
      fclose(f);
   }
}
 
 
int main( int argc, char *argv[] )
{
   GLfloat *buffer;
 
   /* Create an RGBA-mode context */
#if OSMESA_MAJOR_VERSION * 100 + OSMESA_MINOR_VERSION >= 305
   /* specify Z, stencil, accum sizes */
   OSMesaContext ctx = OSMesaCreateContextExt( GL_RGBA, 16, 0, 0, NULL );
#else
   OSMesaContext ctx = OSMesaCreateContext( GL_RGBA, NULL );
#endif
   if (!ctx) {
      printf("OSMesaCreateContext failed!\n");
      return 0;
   }
 
   /* Allocate the image buffer */
   buffer = (GLfloat *) malloc( WIDTH * HEIGHT * 4 * sizeof(GLfloat));
   if (!buffer) {
      printf("Alloc image buffer failed!\n");
      return 0;
   }
 
   /* Bind the buffer to the context and make it current */
   if (!OSMesaMakeCurrent( ctx, buffer, GL_FLOAT, WIDTH, HEIGHT )) {
      printf("OSMesaMakeCurrent failed!\n");
      return 0;
   }
 
   render_image();
 
   if (argc>1) {
#ifdef SAVE_TARGA
      write_targa(argv[1], buffer, WIDTH, HEIGHT);
#else
      write_ppm(argv[1], buffer, WIDTH, HEIGHT);
#endif
   }
   else {
      printf("Specify a filename if you want to make an image file\n");
   }
 
   printf("all done\n");
 
   /* free the image buffer */
   free( buffer );
 
   /* destroy the context */
   OSMesaDestroyContext( ctx );
 
   return 0;
}

The code is actually pretty straight forward. Just have a look at the main function. Create an OSMesaContext, allocate a buffer and call OSMesaMakeCurrent with the new buffer. After that you can call openGL functions and have fun. If you are done destroy the context and free the allocated buffer. Now hit CTRL+O to save it and leave nano with CTRL+X.

3. Build executable and run the test​

To create an executable program run the following command:

gcc osdemo.c -lGLU -lOSMesa -lm -o osdemo32

And try it out:

./osdemo32 test.tga

If everything went right you will see something like this:

and the output image looks like this:

That's it! Rendering openGL stuff without x11, a physical display and gpu!

The next updates will add a proper thread for the stiching process which currently blocks the Ui till it's done. Anyways grab it from github.

This is basically the size i managed to put together. The current release only manages half of the size because of Qt's QImage limits:

Here's a pic with a bit more resolution (i won't upload a full scale image cause it will make my page super slow...):

Additional hint: Stiching atm only works if you have downloaded all tiles and restart the app and point to the output directory again. (update will fix this issue)

So how can you solve this?

The answer is very easy:

First switch your input textures to sRGB:

//this texture will stay uncorrected if it is non linear
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB8, width, height, 0, GL_BGR, GL_UNSIGNED_BYTE, data);

//this one will be corrected on texture lookups
glTexImage2D(GL_TEXTURE_2D, 0, GL_SRGB8, width, height, 0, GL_BGR, GL_UNSIGNED_BYTE, data);

Second switch your final color output to sRGB:

//if you use GLFW
glfwWindowHint(GLFW_SRGB_CAPABLE, TRUE);
//...
glEnable(GL_FRAMEBUFFER_SRGB);


//if you go full hardcore
//use glRenderbufferStorage() with GL_SRGB instead of GL_RGB
//for your final renderbuffer that uses the final non-sampleable framebuffer
//and dont forget
glEnable(GL_FRAMEBUFFER_SRGB);

Thats all you need. If you want more information about gamma correction check this link: GPU Gems 3 Chapter 24

And here's a screenshot of gamma correction in action (with post processing noise, fxaa and sharpen filter):

This is the skybox side's input texture:

Let's have a look on the code:

The complete scene is rendered and stored in a framebuffer and bound as a texture (l_tex) for the following shader. The output buffer is bound properly (def_e in the fragment shader)...

//render a full screen quad with vertex coordinates:
//-1.0, -1.0,  0.5
//-1.0,  1.0,  0.5
// 1.0,  1.0,  0.5
// 1.0, -1.0,  0.5
//or similar

//vertex shader, nothing special here...
#version 400
layout(location=0) in vec4 vp;
void main () {
	gl_Position = vp;
}

//fragment shader, the interesting part!
#version 400
uniform sampler2D l_tex;  //the rendered framebuffer containing the color bound as sampler2D
uniform vec2 win_size;    //the size of the window/framebuffer

layout( location = 0 ) out vec4 def_e; //the manipulated color output

//////////////////////////////////////////////////////////////////////////////
//FXAA
#ifndef FXAA_REDUCE_MIN
    #define FXAA_REDUCE_MIN   (1.0/ 128.0)
#endif
#ifndef FXAA_REDUCE_MUL
    #define FXAA_REDUCE_MUL   (1.0 / 8.0)
#endif
#ifndef FXAA_SPAN_MAX
    #define FXAA_SPAN_MAX     8.0
#endif

//optimized version for mobile, where dependent 
//texture reads can be a bottleneck
vec4 fxaa(sampler2D tex, vec2 fragCoord, vec2 resolution) {
    vec4 color;
    
  vec2 inverseVP = 1.0 / resolution.xy;

  vec2 v_rgbNW = (fragCoord + vec2(-1.0, -1.0)) * inverseVP;
  vec2 v_rgbNE = (fragCoord + vec2(1.0, -1.0)) * inverseVP;
  vec2 v_rgbSW = (fragCoord + vec2(-1.0, 1.0)) * inverseVP;
  vec2 v_rgbSE = (fragCoord + vec2(1.0, 1.0)) * inverseVP;
  vec2 v_rgbM = vec2(fragCoord * inverseVP);

  vec3 rgbNW = texture2D(tex, v_rgbNW).xyz;
  vec3 rgbNE = texture2D(tex, v_rgbNE).xyz;
  vec3 rgbSW = texture2D(tex, v_rgbSW).xyz;
  vec3 rgbSE = texture2D(tex, v_rgbSE).xyz;
  vec4 texColor = texture2D(tex, v_rgbM);
  vec3 rgbM  = texColor.xyz;
  vec3 luma = vec3(0.299, 0.587, 0.114);
  float lumaNW = dot(rgbNW, luma);
  float lumaNE = dot(rgbNE, luma);
  float lumaSW = dot(rgbSW, luma);
  float lumaSE = dot(rgbSE, luma);
  float lumaM  = dot(rgbM,  luma);
  float lumaMin = min(lumaM, min(min(lumaNW, lumaNE), min(lumaSW, lumaSE)));
  float lumaMax = max(lumaM, max(max(lumaNW, lumaNE), max(lumaSW, lumaSE)));
  
  mediump vec2 dir;
  dir.x = -((lumaNW + lumaNE) - (lumaSW + lumaSE));
  dir.y =  ((lumaNW + lumaSW) - (lumaNE + lumaSE));
  
  float dirReduce = max((lumaNW + lumaNE + lumaSW + lumaSE) *
                        (0.25 * FXAA_REDUCE_MUL), FXAA_REDUCE_MIN);
  
  float rcpDirMin = 1.0 / (min(abs(dir.x), abs(dir.y)) + dirReduce);
  dir = min(vec2(FXAA_SPAN_MAX, FXAA_SPAN_MAX),
            max(vec2(-FXAA_SPAN_MAX, -FXAA_SPAN_MAX),
            dir * rcpDirMin)) * inverseVP;
  
  vec3 rgbA = 0.5 * (
      texture2D(tex, fragCoord * inverseVP + dir * (1.0 / 3.0 - 0.5)).xyz +
      texture2D(tex, fragCoord * inverseVP + dir * (2.0 / 3.0 - 0.5)).xyz);
  vec3 rgbB = rgbA * 0.5 + 0.25 * (
      texture2D(tex, fragCoord * inverseVP + dir * -0.5).xyz +
      texture2D(tex, fragCoord * inverseVP + dir * 0.5).xyz);

  float lumaB = dot(rgbB, luma);
  if ((lumaB < lumaMin) || (lumaB > lumaMax))
      color = vec4(rgbA, texColor.a);
  else
      color = vec4(rgbB, texColor.a);
  return color;
}
//FXAA END
//////////////////////////////////////////////////////////////////////////////

void main () {
  //calculate smoothed color by applying fxaa (usually at sharp edges)
  def_e.rgb = fxaa(l_tex,gl_FragCoord.st,win_size).rgb;
  def_e.a = 1.0f;
}

Don't forget to properly bind your framebuffers and disable the depthtest or clear your depth if you use any when you render the fullscreen quad!

Below's a screenshot where fxaa is applied (no zoom left and 2x zoomed right):

Source for shader found here: https://github.com/mattdesl/glsl-fxaa

Why a rewrite and not a port​

The 2012 engine emulated OpenGL ES 2.0 on the desktop. That emulation did not support threading, and threading is what I needed on the desktop side. So I dropped the emulation and moved to real desktop OpenGL. I also wrote my own math classes. They passed 1500 lines of code at 50 percent completion, because I wanted code that I could extend myself. SFML handled window creation and window events.

The code almost used no external libraries. The only exceptions were freeGlut, GLEW and Qt, and I avoided most of the Qt API.

The deferred pipeline​

A deferred lighting pass went in within two days. The picture below shows the point lights.

Ambient light and a directional light came next. Both are additive, and both can be used multiple times.

The frustum culling had a fault, and I fixed it together with some smaller light bugs.

The GPU problem of 2014​

My laptop GPU died in spring 2014. The remaining Intel HD4000 supports OpenGL 3.3 only, and the engine targeted OpenGL 4.0. Two options existed:

• bring the OpenGL target down to 3.3
• get a new machine

A downgrade would have meant a shader rewrite and changed rendering procedures. I bought a desktop machine instead. The engine code then failed to render the loaded textures on the new machine. That fault was small, and I fixed it in August 2014.

Scene management​

The biggest gain came from spatial subdivision. I implemented two octrees, one for static objects and one for dynamic objects. Under average conditions the octrees raised performance enormously.

Alongside the octrees these changes happened:

• restructured most classes
• a CompositeObject class as container for all GameObject related parts
• graphs to measure performance
• memory leaks fixed by moving to smart pointers, and the new bugs caused by that move fixed as well
• bugs in the event system fixed
• octree related bugs fixed
• wireframe and vertex render modes back in the render pipeline
• plans for debug output of light geometry and octree geometry

Debugging the GPU side used AMD CodeXL.

Asset loading and sky​

My hand written OBJ loader got replaced by the Assimp library. The classes were already prepared for multiple formats, so the swap was easy.

Skyboxes add a lot to a 3D scene. Not everything worked correct at that point, but it was close.

Feature list of the renderer in 2015​

• entity management system, work in progress
• post processing render passes
• FXAA, NFAA, sharpening filter, Gauss blur, SSAO
• gamma correction
• Assimp model import
• sky cubes
• improved texture map handling
• octrees
• early view frustum culling
• unlimited amount of dynamic point lights
• directional lights
• ambient light
• deferred rendering pipeline

The shader work lives in its own posts, because those posts contain the code: gamma correction, post processing aa with FXAA and NFAA.

What came after​

The renderer never got a new mobile port. The desktop version reached a usable state, and after that the work moved to other projects. The projects that came before this engine are documented in the old page archive.

Added post processing anti aliaising. Still tweaking it a bit.

The idea behind this shader: NFAA

Update:

Some improvements with less samples per pixel:

And here is a rather high freqency image but with low color luminance varianc:

And here is the same scene but with the edges detected:

Update #2:

I went back to the 13 pixel lookups because testing only horizontal and vertical pixel differences caused on some "edge"-conditions no smoothing at all... Here's my shader if anybody is interessted (Fragment shader):

#version 400
//the final image buffer we want to smooth
uniform sampler2D l_tex;
uniform vec2 win_size;

//the output buffer
layout( location = 0 ) out vec4 def_e;

//the spread of the edge detection algorithm
const float fScale = 3.0f;

float GetColorLuminance( vec3 i_vColor )
{
  return dot( i_vColor, vec3( 0.2126f, 0.7152f, 0.0722f ) );
}

void main () {

  vec2 st;
  st.s = gl_FragCoord.x / win_size.x;
  st.t = gl_FragCoord.y / win_size.y;

  vec2 vPixelViewport = vec2( 1.0f / win_size.x, 1.0f / win_size.y );

  // Offset coordinates
  vec2 upOffset = vec2( 0.0f , vPixelViewport.y ) * fScale;
  vec2 rightOffset = vec2( vPixelViewport.x, 0.0f ) * fScale;

  float topHeight = GetColorLuminance( texture( l_tex, st + upOffset).rgb );
  float bottomHeight = GetColorLuminance( texture( l_tex, st - upOffset).rgb );
  float rightHeight = GetColorLuminance( texture( l_tex, st + rightOffset).rgb );
  float leftHeight = GetColorLuminance( texture( l_tex, st - rightOffset).rgb );

  float leftTopHeight = GetColorLuminance( texture( l_tex, st - rightOffset + upOffset).rgb );
  float leftBottomHeight = GetColorLuminance( texture( l_tex, st - rightOffset - upOffset).rgb );
  float rightBottomHeight = GetColorLuminance( texture( l_tex, st + rightOffset + upOffset).rgb );
  float rightTopHeight = GetColorLuminance( texture( l_tex, st + rightOffset - upOffset).rgb );

  // Normal map creation
  float sum0 = rightTopHeight+ topHeight + rightBottomHeight;
  float sum1 = leftTopHeight + bottomHeight + leftBottomHeight;
  float sum2 = leftTopHeight + leftHeight + rightTopHeight;
  float sum3 = leftBottomHeight + rightHeight + rightBottomHeight;

  float vector1 = (sum1 - sum0);
  float vector2 = (sum2 - sum3);

  //for horizontal and vertical only...
  //float vector1 = -(topHeight - bottomHeight);
  //float vector2 = rightHeight - leftHeight;

  vec3 Scene0 = texture( l_tex, st ).rgb;

  if(abs(vector1) < 0.0001 || abs(vector2) < 0.0001){
    //we don't need to smooth ...
    def_e.rgb = Scene0;
  }
  else{
    // Put them together and scale.
    vec2 Normal = vec2( vector2, vector1);
    Normal.xy = clamp(Normal,vec2(-1.0,-1.0)*0.4,vec2(1.0,1.0)*0.4); //Prevent over-blurring in high-contrast areas! -venzon
    vec2 final_norm = Normal * vPixelViewport * fScale;  // Increase pixel size to get more blur

    vec3 Scene1 = texture( l_tex, st + final_norm.xy ).rgb;
    vec3 Scene2 = texture( l_tex, st - final_norm.xy ).rgb;
    vec3 Scene3 = texture( l_tex, st + vec2(final_norm.x, -final_norm.y) * 0.5f ).rgb;
    vec3 Scene4 = texture( l_tex, st - vec2(final_norm.x, -final_norm.y) * 0.5f ).rgb;

    //Final color smoothed
    def_e.rgb = (Scene0 + Scene1 + Scene2 + Scene3 + Scene4) * 0.2;
    //show the normals of the smooth pixels
    //def_e.rgb = normalize( vec3(Normal.x, Normal.y , 1.0f ) * 0.5f + 0.5f);
    //show  pixels where the smoothing is applied to
    //def_e.rgb = vec3(1.0,0.0,0.0);
  }
  def_e.a = 1.0f;
}

Where the project comes from​

The idea of a task manager existed months before the S^3 version. The first attempt was based on Python and PyS60. That attempt became too time intensive next to my university studies, and PyS60 does not run well on S^3. The S^3 version used the C++ engine instead. A video of the older attempt existed, but unfortunately it got lost in time.

In April 2012 I started the new project on top of the 2012 engine.

Process control​

Task reading and task closing worked in April 2012. Killing a process needs a certificate with the PowerMgmt capability. Without that certificate the thread panics and the app exits. The planned Task Settings window would have offered these actions:

• send system events like EApaSystemEventShutdown
• kill the thread of a task
• read CPU time, priority and flags

Two problems blocked me. First, my knowledge of Symbian C++ was not enough to capture the menu key long press event inside Qt. Second, the 3D window refused to work after the Qt 2D content was opened. The menu key was also unusable outside the application, so I used the proximity sensor to bring the app to the foreground.

Releases​

Alpha v0.11, unsigned, April 2012​

Implemented: the GUI, thread priority setup, process priority setup, killing tasks. The 3D window was not part of the alpha.

Early 3D gui to list all running tasks:

Alpha v0.20, unsigned, June 2012​

New features:

• main window with CPU usage in percent for every task
• long press menu in the main window
• bigger buttons
• colored buttons and attribute values in the task setup window
• system info window with drive stats and a CPU usage graph

The tiles update themselves, and the proximity sensor brings the task manager back to the foreground. What remained was scrolling the tiles, making them touchable, and connecting the 3D app to the working 2D app. I also asked at this point whether the app should be free or paid, and I added a donation option.

Version 1.0, September 2012​

Version 1.0 went to the store. The submission failed first, because the permission to read the C:\ drive was missing, where the settings file and the logs were stored. That caused a fatal chain reaction in the graph drawing.

Switcher passed the Ovi Store quality assurance on 2012-09-19, and it became available on 2012-09-25. The store listing existed at http://store.ovi.com/content/313719. The 100th purchase happened in November 2012.

Bugs on the platform​

Out of memory in the Qt UI code​

This bug deserved its own post. See the QPixmap bug for the broken code, the working code and the reasoning. Short version: scaling a QPixmap leaks memory on Symbian, and converting to QImage before the scale fixes it.

Custom fonts in modded ROMs​

One bug appeared on devices with custom fonts. It looked like modded ROM fonts were the cause, yet the app worked once on a device with modded fonts. The error was QSymbianLeaveExceptionKrerralreadyExists, and it appeared in all applications. Other people had the same problem, and the Qt developers at Digia had an exception in the basic Symbian implementation without knowing the reason for the failure.

The 3D UI rewrite​

The UI of the first release was bad. The new UI was rendered in OpenGL ES 2.0 without any Qt element. A preview animation showed the idea in December 2012, see the UI preview video.

The design work in March 2013 started on paper, then it became a digital layout with metrics. Red areas were touch areas and buttons. Green areas were dynamic information rectangles. A yellow area was a simple overlay.

Almost every UI element used one single image as texture atlas. That choice lets the user modify the whole look by replacing one file.

In the code the base classes came first, and the higher level classes like buttons, lists and switches came from them. Elements were detected by touch gestures in 3D space. The engine emulated OpenGL ES 2.0 on my desktop machine, so everything on the screen looked 100 percent identical to the phone.

The engine project got its own repository, and the phone got started for the first time in May 2013. The app needed a few seconds at the start, and after that it ran like on the desktop machine. The demo showed a draggable green formation that springs back on release, and the exit button already worked. The demo is still downloadable: switcher.zip.

Source code and the end​

I published the source of Switcher v1.0 in October 2013.

• Switcher source: github.com/AJ92/Switcher

Nokia's Symbian OS died, and Switcher with it. The old Symbian downloads with the .sis files are gone.

Spectre, a music visualizer​

Spectre is a small music visualization program. It is powered by the FMOD Ex sound engine, and the GUI uses Qt. I was surprised how easy FMOD is to use and to combine with Qt. The project was part of the media subject in my second semester.

Two archives exist: spectre.zip and the source spectr_source.zip.

Wochenplaner, a weekly schedule manager​

The tool manages a weekly schedule. The features include a printable schedule and a SQL database. The GUI uses Qt, because any object oriented language was allowed. The server is a simple Windows console application that runs as a CGI application on an Apache server with a MySQL database. The server code uses only the standard Windows libraries and the MySQL libraries from Visual Studio.

The project never went beyond that state, so no download and no source exists for it.

First output​

The graphical part reached a first release in March 2012. Physics with the Bullet engine, audio and touch input were planned for a later date. A level editor and a shader editor were planned as well.

Draw call and buffer management​

All meshes are sorted by material. This prevents multiple texture and shader switches per frame. Identical meshes are sorted next to each other, so the vertex arrays do not get send to the GPU again.

The debug window below shows CPU and GPU load, framerate, timestep, both averages, material switches per frame and vertex array switches per frame. The engine ran on one CPU core only.

Vertex buffer objects replaced the vertex arrays and the gain was large. The numbers below are triangles per second.

• about 34 560 720 triangles per second without VBOs
• about 115 200 720 triangles per second with VBOs

The test machine used an Intel Core i7 at 2.2 GHz and an nVidia GT 540. The Nokia N8 also gained a lot of speed.

The phone test reached about 1.5 million triangles per second. That equals 48 012 triangles in 33.3 frames.

Shader class rewrite​

The shader class got a full rewrite, and the renderer adapted to the new class. After that change it became possible to write own shaders and own rendering techniques. The cost was the reimplementation of the shader class with a few own functions.

Runtime object and texture control​

The early release could not delete models, meshes, materials or texture maps at runtime. That became possible later, and these were the rules I had to respect:

• load a model once and create instances of it, or load a full copy into RAM
• update single texture maps of a material at runtime
• delete a model when it is no longer needed
• never update or delete the real model when instances exist, because that changes every instance

Engine tools and the model viewer​

I started a tool window in April 2012. The first version held system stats and window tools only.

In September 2012 I built a small model viewer to preview models inside the engine and to play around with shaders and settings. The model is Betty, the first ship of Gallaxy on Fire 2. A strong specular map makes the scratches shine and the hull starts to look like metal.

Procedural planets​

In april 2010 I got inspired by the amazing tech demo that later became Infinity: Battlescape

I played around with pocedural generation of spherical shapes. The first step of was the vertex generation of an icosahedron sphere. Subdivision level 2 gives about 0.9k vertices. Subdivision level 5 gives over 61k vertices.

The experiment never went any where far, but later I tried to pick up the idea again in unity, but also quickly dropped it.

First release​

The manual and the class diagram went along with the graphical part. The release also used the Imagination Technologie emulator for Windows, which was the way to develop for the phone from a desktop machine. The code was opensourced and ultimately ended up later in the following repository: https://github.com/AJ92/Engine

The 3D part of a future project Switcher used an early version of this engine and went out for testing in May 2012 as an unsigned app. The https://github.com/AJ92/Switcher repository still contains parts of this version...

Planned, never shipped features​

The plan for the next engine version listed these items:

• model viewer with ASE, OBJ, FBX and a binary model format loader
• binary model format exporter
• shader tweaking and shader coding at runtime
• model variable editing at runtime, for example light parameters
• Bullet physics 2.8x
• an Android port where everything gets written once again

The engine never reached that state. Symbian died in 2013, and the desktop rewrite got a new name. See the Quark renderer for the follow-up project.

On 2012-08-17 the replacement arrived, and development could go on. This was the hardware change:

The same problem appeared again in 2014, when the laptop GPU died and the Intel HD4000 left with OpenGL 3.3 only. See the Quark renderer post for more info.