You may notice that this image bears some extraordinary similarities to the final one from my last post., which was basically the output of the OpenGL tutorials. For the record, I did not just change the texture and draw more blocks here! A quick summary of what this screenshot represents:
paring down and copying most of the boilerplate and base abstractions from the tutorial into a new file in my existing c minecraft project, without most of the actual scene, the point lights, etc.
integrating the pre-existing world generation and perlin noise code with that new boilerplate without just inlining it all into the big new file
rendering the “state of the world” out of a newly-added gamecontext world-array, and paring its size down for performance reasons until I get a display list and frustum culling written
getting all that stuff to compile
So there’s a tremendous amount ahead — cubemapped textures for the blocks, frustum culling, and basic Quality of Life stuff are all up next so I can start drawing the full 100x100x100 map and having it look halfway decent. I’ve had to move frustum culling way up because at full size things just would not draw. But this is lovely progress for about one day into trying this plan.
A last thought here — it seems almost like wordpress is applying a filter to my screenshots? They’re not particularly bright on my laptop, but at some point during upload it feels like the colour washes out a little, which robs them of the peakglory they so desire. Maybe it’s just in my mind. They’re still nice.
Years ago, for a university project, I made a minecraft-like game in C with some already-very-dated OpenGL toolkits and versions. It was a color-by-numbers affair: the professor would give us a big code dump and we’d hack on it and add some features, then the next assignment the professor would give us the same code as before but with more in it, and we’d hack on it and add some features. Everyone ended up with something that roughly looked and felt like minecraft. And for the record, the new code we were given each time around was atrociously bad.
I plopped my project up on github and left it behind in ~2011, briefly returning in ~2018 or so out of curiosity. It was in a state where it rendered ~5 frames per second, motion looked laggy and stilted, digging meant a random block somewhere around the user might disappear, the lighting as the player moved around or the sun moved through the sky constantly broke and flickered, mobs (monster objects) under the earth didn’t appear correctly at all, and multiple users could connect to a server over the network, but they could not see each other or interact in any real way. Also, the mouse would jump around wildly when you first loaded the game or if you minimized it. And the mouse doesn’t capture in the window. And there were no textures, and all the code was randomly formatted.
So, it was a bit of a fixer-upper! It did still mostly compile, and it roughly looked like Minecraft, in that it was a bunch of grass-colored cubes shaped by Perlin noise, with an underground cave system.
During a recent extended stint of free time, known colloquially as funemployment, I decided to pick this old project back up and see what I could do with it. So far, I’ve:
fixed the framerate issues (turns out we weren’t culling right)
fixed the lighting flickers (something with normals. I pretty much just turned off specular light.)
added a simple super hacky crosshair (a few quads in an orthographic matrix view!)
made networked players able to see each other as they move around, and fixed a lot of little networking bits around e.g. what happens when players come and go from a server
formatted all the code using an automated tool, AStyle (which takes some hacks to make work on Arm64 macs in sublime!)
made motion and falling much smoother, based around acceleration instead of per-update direct changes, invoked upon each keyboard-polling function call, at whatever speed the OS registers repeated key-presses and ONLY when the OS has registered keypresses
mobs are now actually visible, rendering in the right space, and wander around in the caves bumping off of walls
fixed up some major, wacky inconsistencies in the co-ordinate system like some co-ords being stored as negative values and some others as positive (this is sometimes known as “the camera problem” in graphics — to move the “player”/camera X units in a direction through the world, the programmer really moves the world itself X units past the player in the opposite direction. This lends itself to storing the camera’s position as negative co-ordinates in many naive setups, and it can make for a lot of confusion!
Fixing all of this in the past month has been a good bit of fun. The largest remaining issue is to bring in textures, along with better-looking lighting, and that has necessitated a journey. Once I got the “C Minecraft Clone” project to the moderately acceptable state above, I left it temporarily, to work through the Learn OpenGL tutorial series, with the end intent of returning to rewrite my C Minecraft Clone usingmodern OpenGL!
Who, or what, is Modern OpenGL?
My existing project is written using Ye Olde OpenGL, roughly Version 2.1, which uses a flavour of OpenGL API referred to as “immediate mode.” This mode includes many built-in concepts of lighting, materials, and matrix manipulation, which can reduce the amount of boilerplate required to get the basics up and running. This API usually has calls to functions like glPushMatrix, glShadeModel, glTranslatef, glRotatef, glMatrixMode, glLightModeli, and other similar beasts. Because it does a lot of work for you, there’s a lot of magic going on under the hood that makes it hard to use correctly, as in both efficiently and with nice looking scenes. And there’s unfortunately not a lot of good resources out there updated in the last decade for understanding it.
Modern OpenGL, circa Version 3.1+ (I’m using 4.1), does a lot less for the user out of the box, but gives (er, you might say “foists upon”) the programmer a lot more fine-grained control. It exposed what is referred to as the “core mode” API, in which you must define buffers, set modes, make glDraw calls yourself, and write shaders to render everything, using the C-ish shading language, GLSL. You need to at least write a Vertex Shader and a Fragment Shader, which handle steps in the rendering pipeline to transform your abstract vertices into rasterized pixels. All of the Learn OpenGL tutorial series linked above teaches how to write and think in “core mode” OpenGL, and it gives a great introduction to writing shaders — so working through these to understand how to use it has become my recent project.
For the record, OpenGL 3.1 came out in 2009, while OpenGL 2.1 is from 2006! Neither is a spring chicken. But the code as currently written is using stuff that was deprecated nearly 15 years ago, while OpenGL 3.1+ is still pretty much “how OpenGL is used today”. A further tricky aspect of this landscape is that there isn’t necessarily one line in a program that says what version of OpenGLwe’re using. There is in some programs, but not all! When you don’t have a clear answer, you’ve got to check what OpenGL version your Operating System and Drivers have access to, which is what the program will use when run.
Even then, you could have OpenGL 4.1 installed, but if your program uses immediate mode APIs it will run in an odd OpenGL 2.1ish compatibility mode. You pretty much need to look at what sort of API the program is using — the functions from the first paragraph above are telltale signs of some very dang old graphics code.
“Wait, isn’t OpenGL over? Shouldn’t you be using Vulkan?”
Yeah, well that’s, hey listen buddy — I’m not some kind of magician over here! I’m just learning things, alright? Sure, I found the Vulkan tutorial, and I might look at that down the road too, but it’s even lower level than core mode OpenGL. There’s also ongoing weirdness where (of course) Apple has thought different, and created Metal, which is its own whole world, and because I’m using an M1 mac, the Vulkan vs. Metal story might be a saga.
So for now, I just want to take some baby steps to a game that looks a little better. That brings us to the problem: I have a big unwieldy tutorial project with its shaders and vertex buffers and all the goodies, and I have a big old dinosaur Minecraft project which is written totally differently, and — oh right, GLUT vs GLFW! That’s one of the biggest differences, and…
GLUT and GLFW are wrappers around operating-system level concerns like creating or disposing of a basic window, and receiving mouse and keyboard input. They’re platform-independent APIs that aren’t meant to do “the OpenGL parts” for you, moreso “the messy OS stuff,” and using something like them as a framework to initialize an OpenGL program is widely regarded as A Good Idea.
But GLUT is deprecated. GLUT is so deprecated that the GLUT Website literally just says, “We direct you to use FreeGLUT found on SourceForge: http://freeglut.sourceforge.net/. The original GLUT has been unsupported for 20 years.” SourceForge always sets off my spidey-sense, but of course my 2010-era webpage is written using GLUT. At least I got it to compile!
GLFW is what the Learn OpenGL tutorials I’ve mentioned so far use, and it’s also what the Vulkan tutorials use. It has been a lot nicer on the eyes and the mind than GLUT is, so I’m sticking with it. It was pretty easy to install, has recent-ish docs around, and modern amenities abound. There are alternatives, but it does really seem close to the consensus framework. Unfortunately, the two systems are pretty different:
GLUT uses callback functions extensively, but relies on global variables for communicating with them
GLUT and GLFW each have distinct sets of functions to create and set up a window
GLUT pushes toward using its keydown function (and later releases added a keyup function), in which the end user is likely to directly respond to key events and would have to build their own keymap abstraction, while GLFW provides an extensive keymap abstraction that can be polled at any time
their mouse handling seems overall pretty similar
glut includes some utilities closer to the opengl drawing layer that glfw doesn’t go near, for example, a main loop callback
Where were we… Right! Because of all this, the game and tutorial have very different shapes, which makes it hard to make any incremental steps here.
A Tale of Two Plans
Of course I have all of my stuff stored in git, so I’ll be working on a branch and trying things out with low risk, but I’d like to nail down a direction for work. I sat down today with all my new OpenGL knowledge and re-read my C Minecraft Clone code, and I’ve come up with two overall plans of attack, but I’m undecided on which to take. So we’ll do it live. They boil down to “rewrite the original with new OpenGL hotness” vs. “port the unique bits of the original into a new modern OpenGL project”.
Here’s a rough sequence of discrete steps I could follow to “rewrite the original”:
Remove all lighting from the C Minecraft Clone, because it’s complex and gonna change completely.
Swap from GLUT to GLFW for initialization and drawing, and drop all the keyboard and mouse controls.
Re-add keyboard and mouse and window-resizing controls using GLFW.
Rewrite the core display function to use a new cube vertex list with texture and normal co-ordinates and some basic shaders, and just rgb-diffuse-color materials.
Rewrite the player and mob display functions similarly.
Figure out how to port the frustum culling over.
Add textures and lighting back in, using the new OpenGL shaders.
Meanwhile, the “port everything over to a modern project” rewrite could go like the following:
Set up a GLFW project with basic shaders, abstractions, and FPS-like controls, taking heavily from my tutorial project.
Port over the world-generation and rendering of cubes to form a map.
In turn port over controls, digging, mobs, and networking.
Figure out how to port the frustum culling over.
Add lights and fancy shading where it fits.
The first path feels to me like it will be harder. But, despite the very ship-of-theseus nature the project has, it feels like a more pure idea of “rewrite of the project using modern opengl”, which is the goal I have felt an emotional draw to all along. It rips the OpenGL bits out of the existing project and upgrades them, rather than just starting with a new project and reimplementing everything that made the project unique (via porting it all over).
So, which direction will I go on this? At the moment, I’m a little torn. I’ll continue to mull it over this evening, and some whim will take me forward to try soon.
Were this a “work project”, I think I would definitely take the second path — I can pretty much get a working “modern OpenGL” project running immediately, and then each porting step is fairly small and self-contained and they continuously build the project up. By comparison, in the first path I need to rip out working code to reduce its capabilities and then restore them on each of Steps 1, 2, 3, 4, and 5. But it isn’t the work I thought I was planning / preparing to do — isn’t that a funny concern? I think the best path is becoming pretty clear.
Just for a reference, here’s roughly what my OpenGL tutorial project looks like today — the textures are mostly supplied by the tutorial itself, and the big “advance” is the lighting (plus having textures at all):
Thanks for reading. I hope you’ll join me soon for my next post, “How I failed to make any progress rewriting my old school project in OpenGL!”
I googled “distributed systems masterclass” on a lark today, and ran across a nice looking free course (no affiliation), and it got me on a bit of a roll for related content. I found some new stuff and I dredged up some old favourites. Once I had those, I began writing an email to link it to some friends and I thought: why not put that in public!
So here’s a little bag of links. It’s kind of videogame focused, and I should note up front I have not watched/interacted with all of this yet; there’s some inline notes about what I’ve actually engaged with.
(All of these links are set to ‘open in a new tab’, because I personally want to destroy your productivity.)
The course I just found: https://www.distributedsystemscourse.com/ — the topics look interesting and like a decent mix of intermediate subjects, and the actual content seems well made at a glance, and it’s free! This seems like a rare find to me.
I jumped right to checking out the video for How Counterstrike Works (a.k.a. Time in Distributed Systems), which really got my goat on its mix of topics. It looks like it’s got some actual meat to it, but it isn’t some 3 hour outing in the minutiae of one project or a textbook chapter.
The classic Leslie Lamport paper, Time, Clocks and the Ordering of Events in a Distributed System (PDF), which I have tried and struggled to read a few times in the past, because (if I recall correctly) I got bogged down in the formal bits. I found this blog article today when googling the name of the paper, which is a kind of “reading the paper guide”. It has some summary content and has been quite readable so far. I’m hoping it will help me form a useful mental image to take back to the actual paper and hold in mind when working through the formal details.
Leslie Lamport’s website — holy moly! How have I never been here? There’s so much commentary here, beyond the riches of the list of papers he’s hosted for direct access. What a trove.
Indenting for a brief aside: I’ve always been a sucker for commentary. In the many Asimov essay paperbacks we had around the house growing up, I far more enjoyed the little snippets of preamble and afterword commentary he’d include than the essays themselves. I felt those bits were treats doled out in exchange for traversing the real content. Asimov would finally stop teaching me for a moment to talk to me, and so much life and personality would jump out and bind flesh to the essay’s bones by opening a keyhole view to the living essayist behind the curtain. His tone in the essays was always personable, but those extra bits let me imagine Asimov as a human I could know, and let me connect to the content as if it were from a friend. I should find some of those books.
(… how better to show I’m a dilettante without saying “I’m a dilettante” — anyway!)
The classic “1500 Archers” article, which I read 15 years ago and keep meaning to go back to. If I recall my partner’s feelings on it accurately after she read it a year or two ago: if you’re just starting out in understanding multiplayer networking, it gives you enough to know you have no idea what’s going on and that your game does not work like this. It’s lacking in direction on how to rework an existing game to run as a deterministic discrete simulation built atop an abstraction of turns, so if you don’t already have/know how to do that, you’re kinda boned on it helping you.
This amazing GDC talk, Overwatch Gameplay Architecture and Netcode, which seems to be unlisted these days, where a senior Overwatch developer walks through their ECS architecture and does a bit of a deep-dive netcode presentation. The descriptions he gave of prediction and fallbacks made a lot of things settle into my mind in a way other content hadn’t previously.
Related, but I’ve never watched it — Networking the Gameplay of Halo: Reach. This looks like a really similar kind of video to the above, but focused on netcode. I found this while wasting 30 minutes searching YouTube for the Overwatch one, and I’d like to watch this someday.
If anyone ever ends up reading this: I’d love to see your distributed systems links. No videogame networking focus needed!
Edit: and here’s some BONUS LINKS of VARYING or UNKNOWN RELEVANCE and QUALITY
I don’t walk every day, but I do consider myself a walker. When I lived in Toronto, I would often walk from Bloor and Dundas West up toward Sheppard, catch the subway back down to Bloor, then walk west to my home around High Park. At a different time I lived near Victoria Park in the east and would walk from Yonge and Dundas out there after work most nights. Some of my longest began downtown, looped out east of the beaches, north up to Eglinton, and then back to High Park or St Clair West, depending on the time of my life.
A good friend of mine also grew up walking quite a bit, and a few years ago noted how, as a game developer, he’d acquired an interest in understanding the experience of traveling from town to town by foot — he asked if I’d be up for a fall full-day hike from the city of Guelph to Burlington, about 40-50km away, and I was all in. It was a 12 hour walk and at times seemed pretty rough, but it started a tradition. We’ve done 4 or 5 of those in the intervening years, mostly closer to 30km or so, and with some disrupted years around the pandemic (couldn’t rely on rest stops being open!) Each time they’ve been a magnificent experience.
Not all walks have to be epic! I regularly wander the neighbourhood with my partner and we’ll just go out and tramp about on the local roads for an hour. We really get a sense of the place we inhabit and of the life which inhabits it, of the connection between places, and of the places in between.
So anyway, I like to walk around, and I ran across this recent post by Derek Sivers, which focuses on a big week-ish/100km-ish long walk through Thailand with a group of ~10 people, keeping a leisurely pace and having time to take in the environment and to just chat with interesting folks each day. Putting my pangs of jealousy for those who lead such charmed lives aside for a moment, I’m truly heartened to see this kind of thing happening somewhere; it feels like the kind of stuff we could all benefit from more of. I’ll definitely be thinking in the future about how I might make something like it happen someday with some old and new friends of my own.
Then after the post, I ran across many great future-consideration walk suggestions in the comments. There were a ton of comments and I didn’t read exhaustively, but I’ve copied a bunch here which stood out to me as I glanced around. Initially I just opened a text file for this, but I figure: why not put it in public? Maybe someone else wants a quick list of ideas.
Santiago de Compostela (Spain)
West Highland Way (Scotland)
Coast to coast (England)
Lake district (England)
Inca Trail (Peru)
Kilimanjaro (Tanzania)
Tour de Beaufortain (France)
Les Ballons des Vosges (France)
Camino de Santiago
The Rota Vicentina and the Fisherman’s trail in the south west of Portugal
Te Araroa, maybe in Marlborough or Northland in New Zealand
Mt. Rinjani – Indonesia – 4-5days – might be more intense than desired
Machu Picchu Inca Trail – during the offseason, not a lot of people, up to 7 days, not super intense
Teyuna (“Lost City”) in Colombia
Serra Tramuntana, Mallorca
Cheers to Derek for sharing about the idea/experience and thanks to posters CRIS, Chema, Auke, Lawrence, Lale, Thomas, Chip, BT, Jorg, and any whose suggestions I copied but forgot to link for sharing their ideas.
We’ve been playing the NES we purchased earlier this year a lot lately. While we do have an Everdrive ready to help us get some homebrew going later (I’ve worked through a fair few of the famicom.party tutorials so far) and to play some harder-to-find titles, it’s all been retro-game-store-bought original cartridges so far, which is why it was a little concerning when the wicked music of The Legend of Zelda noticeably pitched down a few times during gameplay last night. The sprite animations all got a bit gooey and molasses-like, and it felt like the system was really struggling. Once would be spooky enough, but this happened repeatedly!
It turns out, that’s apparently normal when the system is showing a lot of sprites. I’m surprised I’ve never noticed / didn’t know this already, but then, how much time have I spent playing sprite-heavy NES games on the original hardware? I’m doing it now, and it’s happening now. People on reddit mentioned that not having this happen is one of the best aspects of using an emulator, but one poster linked to the above post: how to avoid it on the original hardware by modding your NES to run at a higher frequency! I don’t want to go messing with our dear grey friend, but someday this might be a very fun project. We’ve already disassembled it to clean and pull the pins, so what’s left but to start adding components and resoldering things!
While I’m near the topic, here’s an obligatory link to Tom 7’s Reverse Emulation and Harder Drive videos, some of the best nintendo-related nerd content I have ever seen. Or will ever see.
