Weekly Summary Jan. 20 - Jan. 26

https://wmsalazar.nfshost.com/game is the full URL for the game I put online last week. I continued working on it this week: it now includes all of the functionality of the original game.c program with almost none of the inaccessibility. If you click the link, any WebGPU-conformant browser should just immediately start loading it on its own.

game.js uses a static 1000x1000 canvas element which doesn't scale automatically to fit the page; I've noticed that on some computers this mandates manual user intervention to be able to see the entire playing area. It also (naturally) doesn't work on browsers which don't have WebGPU enabled; at the time of writing, Firefox and Safari seem to be the only major browsers not to enable WebGPU by default on most systems (although there also seems to be limited compatibility with Linux pretty much everywhere). These are the only serious remaining accessibility problems I've noticed so far. I spent some time this week researching how to make it work on mobile phones and it seems to be doing so very well now; it also continues to work completely fine on PCs from what I can tell.

As expected, I've now completely finished the first fully-working version of game.js this week, and have now somewhat run out of things to do. I'll likely spend much of next week focusing exclusively on reorienting myself. As always, thank you for reading.

Weekly Summary Jan. 13 - Jan. 19

I implemented a majority of the underlying functionality of game.js (a WebGPU clone of game.c) this week—although it's not 100% done yet, the general controls are now working. I've uploaded the current version to this website's /game directory, click here to see it. I'll likely continue to update this page as I progress.

While functioning, there are still a couple fundamental issues with it. So far I've focused exclusively on making it work on PCs; it seems to be running very poorly on mobile devices at this time. At the time of writing it also doesn't report failure if WebGPU is unavailable on the user's browser. (Check here to see if you're using a browser with WebGPU enabled.) And, of course, there are a number of features I simply haven't finished yet. The 'TODO' comments scattered throughout the program's source code specify what I still have left to implement.

Despite not finishing game.js this week, I'm still more-or-less happy with the progress I've made on it. I'm fairly confident that I'll have it completely cleaned up by the end of next week. As always, thank you for reading.

Weekly Summary Jan. 6 - Jan. 12

Moving forward, I'm unlikely to embed any more WebGPU code directly within this specific page; I'm likely going to upload future programs to separate pages and then link to them. I'm currently planning on recreating game.c (a Vulkan program I made a while ago) as a WebGPU program so that I can upload it to a web page; I don't know how long this will take but I'm confident that my underlying understanding of WebGPU is now sufficient and I'm hopeful that I can make progress relatively quickly. As always, thank you for reading.

Weekly Summary Dec. 30 - Jan. 5

I continued trying to front-load the process of learning WebGPU this week. As expected, I finished the initial read-through of the HTML specification on Monday; over the rest of the week I also read the entirety of the WebGPU specification and got about halfway through the WebGPU Shading Language specification; I also spent some time on the MDN Web Docs website to try to sharpen up my JavaScript a little bit. I'll likely finish the WGSL specification sometime tomorrow, at which point I'm planning on making some simple applications with it. I'm currently hopeful that I'll have something fully ready to post on this website by the end of next week.

Up until now, I've only written code designed to run on my own personal computer; everything I've posted on this website so far are only saved replays of these programs. WebGPU will make it possible to run these sorts of programs on this website itself. For example, I should soon be able to reproduce the animated triangle from the October 27 post:

The above is simply a video replay of a program running on my own laptop; with WebGPU, I should be able to make a live version of this program which runs seamlessly inside this page itself. I haven't decided whether or not I want to remake this specific program yet but I'm optimistic that I'll have at least something done by the next Sunday.

As always, thank you for reading.

Weekly Summary Dec. 23 - Dec. 29

I don't know if I'm the only one who does this but over the past year or so I've gotten into the habit of reading full specifications before ever actually trying to use the technologies they specify. I think this process makes sense for obscure APIs for which there aren't a lot of tutorials floating around, but for technologies as ubiquitous as HTML it's probably questionable. I definitely might end up regretting it, but (as anticipated) I exclusively spent this week reading the HTML specification and don't yet have anything to show for it. At my current pace I'm probably going to finish sometime tomorrow; hopefully this gamble pays off in the form of increased underlying understanding as I transition into the WebGPU specification. As always, thank you for reading.

Weekly Summary Dec. 16 - Dec. 22

I added a different enemy type and a splash animation into game.c this week:

game.c is now functionally complete with five levels in total. However, I need help actually beating it—I've never gotten past level four. To this end, I've now begun researching how to make a version of game.c that runs entirely in a web browser.

I've taken an interest in WebGPU, which seems to sort of be like a web-page version of Vulkan. However, my underlying understanding of web development is currently, in my opinion, too shallow to jump straight into this, which is why I spent a lot of time this week reading specifications to try to get myself up to speed quickly. I'll likely continue doing so over the next couple weeks without too much to show for it in the short term.

As always, thank you for reading.

December 15 update

I haven't spent as much time working on this project over the past few weeks as I would've liked to, but I've still made some progress. I used the drawing program I wrote about a month ago to make some simple drawings to integrate into game.c:

Although still unfinished, game.c now has multiple different levels (signified by background color) and a win condition:

I also added custom colors into the drawing program. The user can now arbitrarily set the cursor to any color, including transparent ones:

As always, thank you for reading.

Weekly Summary Nov. 11 - Nov. 17

I only made slight progress on game.c this week. It can now show multiple enemies on the screen at the same time:

As always, thank you for reading.

Weekly Summary Nov. 4 - Nov. 10

This week I significantly improved the drawing program from last week. It now lets the user toggle between nine different colors (and an eraser) and change the boldness of the output. It now draws smooth lines.

As before, it saves images into transparent bitmap files:

(I'm using a slightly obscure image format so some browsers might not render the above image properly. If the sun and the pumpkin are appearing on your screen as different shades of light blue, this is probably why.)

I've already begun using this program to draw transparent images to load into Vulkan shaders. For example, I've begun using saved images in the skeleton for a small Vulkan-based video game I've started making:

Each of the digits in the top-right corner of the screen are hand-drawn; their transparency makes them much easier to render over solid backgrounds.

I'm likely going to try to continue making progress on the game next week. As always, thank you for reading.

Weekly Summary Oct. 28 - Nov. 3

Last week I theorized that the vkQueuePresentKHR Vulkan method was passing memory between the discrete GPU (into which lister.c writes memory) and the integrated GPU (which controls what appears on the screen). This week I tested this theory by putting the discrete GPU in charge of the screen and then running lister.c; this was the output for a typical frame:

The gap between the last two lines has increased significantly, but this was expected: Wayland automatically changes its frame rate from 240 (about 0.004167 seconds per frame) to 60 (about 0.016667 seconds per frame) when you switch to the discrete GPU. I assume that the compositor was simply waiting during most of this time which is why I'm not too concerned by it. However, the time it takes to call the vkQueuePresentKHR method (between the third and fourth lines) unfortunately didn't shrink as much as I was expecting it to, implying that I might've been wrong about vkQueuePresentKHR involving a memory pass between GPUs (or at least that the bottleneck this causes wasn't as severe as I'd thought).

I finished a graphics-pipeline version of window.c this week. This is my seventh different implementation of this program, the other six being:

• I wrote the first version with direct Wayland protocol code. The CPU writes every pixel in the window into a buffer object provided by Wayland.
• The second version is the same as the first but CPU-multithreaded: this version divides the job of drawing a frame into twenty separate threads running on the CPU.
• The third version uses GPU multithreading via the OpenCL library to divide the task of drawing a frame into thousands of different threads. Unfortunately, this technique involves passing memory written onto the GPU back into the CPU to be written into a Wayland buffer; on the back end, the Wayland compositor then eventually passes this memory back into the GPU for display.
• I wrote version four at a time when I lacked confidence in Nvidia's implementation of the Vulkan swapchain extension; this version (and version six) actually bypass the swapchain entirely, instead writing frames into Linux DMA memory and then passing these frames into Wayland via the Wayland protocol's Linux DMA-BUF extension. Version four runs on the integrated GPU and version six runs on the discrete GPU; these versions, respectively, only work over dynamic and discrete-only graphics setups.
• The fifth version uses a Vulkan swapchain just like the new seventh version, but, like versions four and six, this version draws the frame via the Vulkan compute pipeline instead of the graphics pipeline.

I took some time this week to add timestamps into each of these seven versions to get a feel for which ones were running fastest and why. Unfortunately there's still a great deal of inconsistency in the speed at which the newer versions of window.c are drawing frames; however, versions five and seven do seem to be drawing frames in almost identical amounts of time. This was surprising to me; I would've guessed that the graphics-pipeline version would've been faster. I was also surprised to see that version five actually seemed to be running consistently faster than version six; I would've thought that using low level memory would've allowed version six to keep pace with the swapchain versions.

For the time being I'm likely to switch between versions five and seven arbitrarily as starting points for new projects. For example, I used version five as a starting point for a very, very rudimentary raster graphics editor I made this week:

This program draws a black line when the user presses down the mouse and drags it over the canvas. It only draws a point once per frame and doesn't automatically connect them (as demonstrated by the dotted lines in the above image); had I thought to use version seven as a starting point instead of version five, I could've passed these points as vertices into a graphics pipeline and programmed a vertex shader to connect them automatically. The interchangeability between versions five and seven gives me the liberty to make these sorts of decisions more-or-less arbitrarily moving forward without having to worry about efficiency trade-offs.

You can save images by pressing the 'S' key; the program saves images onto transparent backgrounds. Here's what the saved version of the above drawing looks like on this web page:

I'm unlikely to spend a great deal of time in the near future looking for ways to speed up existing programs and instead want to focus on writing simple applications like the one above. As always, thank you for reading.

Weekly Summary Oct. 21 - Oct. 27

I've now animated the triangle I made last week:

I decided against learning GLFW this week. My primary short-term goal is to learn the Vulkan API and learning GLFW doesn't support this goal in any way; Wayland protocol code is still a completely sufficient means of generating windows on its own for the time being. Instead, I split my time between finishing the triangle animation and looking for ways to speed up lister.c.

By default, my system displays 240 frames per second; unfortunately, I've yet to create a version of lister.c that consistently draws frames as fast as my system can display them. I spent some time this week trying to get a better feel for where lister was slowing down and why. I created a benchmarked version of lister.c which outputs the following block of text to the terminal once per frame:

The final line of this image ("time since last wayland callback: 0.004245") indicates the amount of time (in seconds) it took lister.c to completely generate and display a particular frame. Ideally, we should be displaying frames once every 1/240th of a second (so about once every 0.004167 seconds); in practice the system lets this number vary by about 0.2 milliseconds in either direction, so this specific frame is very much within the range we would consider normal. However, this output often goes through stretches where it looks significantly worse:

There are significant spikes between the first two and the last two lines, but the largest spike by far is before the fourth line ("queue present called"). During this time, lister.c is calling the vkQueuePresentKHR Vulkan method; my working theory for why this method takes so long is that it involves passing memory back and forth between two completely different pieces of hardware on my computer. I can actually demonstrate this by running lister.c on my integrated GPU instead of on the dedicated one:

Although the first spike (which is the amount of time it takes the GPU to actually draw the image; this slowdown was expected because integrated GPUs are definitively weaker than discrete ones) and the last spike (in which we wait on the Wayland system to actually process the frame) have gotten significantly worse, the spike before the fourth line has almost entirely disappeared, further suggesting that there might be a bottleneck caused by passing memory between the two GPUs. It might be necessary to bypass this bottleneck entirely by configuring my system to use the discrete GPU both for drawing the frame and for controlling the screen (literally just keeping the integrated GPU powered completely off). Fortunately I already know how to do this and will likely experiment with it early next week.

Over the past few months, I've created several different versions of the following window:

This window moves from side to side as the user presses down on the arrow keys. I'd like to create another version of this window using the Vulkan graphics pipeline so that I can compare its performance to that of the previous versions. After that, I'm likely going to stop treating learning the Vulkan API as a project in and of itself and transition into figuring out how to make actual practical use of it.

As always, thank you for reading.

Weekly Summary Oct. 14 - Oct. 20

I've now replicated the triangle from vulkan-tutorial.com:

Although this does require significantly more effort than most people would expect (see vulkan-tutorial.com for details), I actually wasn't expecting to finish it this week.

There's a lot of variation in how different computers manage what appears on their screens. Most computers run either Windows or macOS, both of which have completely separate ways of going about this. Linux systems are also completely distinct from both of these, and there's also variety within Linux itself: some systems use the older but more-reliable X11 windowing system and others use the newer (but still slightly experimental) Wayland protocol. A key divergence my own code so far has made from vulkan-tutorial.com is that my code communicates with Wayland directly whereas the code on vulkan-tutorial.com does so through an API called GLFW. I spent a good amount of time earlier this year learning how to use Wayland directly (mostly because this is just something I was personally interested in knowing how to do) but I'm likely gonna spend some time next week learning GLFW, which is designed to be seamlessly compatible with as many systems as possible.

I haven't decided whether or not I'm actually going to switch to GLFW moving forward, though. I'm apprehensive about potentially losing functionality unique to Wayland. For example, from what I can tell, Wayland is actually relatively unique in that it allows you to draw transparent images. In Wayland, it isn't difficult to make a window which looks like this:

I don't know yet whether or not GLFW lets you do this sort of thing, so I'm not gonna make a decision until I understand it better. Once I'm done learning GLFW I'm likely going to make some sort of animation with the Vulkan graphics pipeline just to continue getting better with Vulkan.

As always, thank you for reading.

Weekly Summary Oct. 7 - Oct. 13

I added the ability to save and reload mid-run to lister.c this week. Previously, the user had no way of restoring their progress if they quit the program before finishing the sort. This is no longer the case: now, lister.c will pick up seamlessly right where the previous sort left off by default.

lister.c uses a background thread to load images into video memory, making it possible for the user to begin sorting images before all of them are loaded. This works fine with a reasonable number of images; however, with a large enough number of images (in the thousands or so), the GPU will run out of memory completely and lister will crash. As a workaround, I created a program called lister_otf.c this week. Instead of loading in every image at once, lister_otf only keeps the images currently displayed on the screen loaded, unloading them as soon as possible. lister_otf.c is usually noticeably slower to transition images than lister.c, but unlike lister.c it works seamlessly with arbitrarily large sets of images.

lister.c uses a Vulkan compute shader to draw every pixel it puts on the screen. GPUs should, in theory, be able to draw images significantly faster than CPUs by dividing responsibility between thousands of different GPU processors (whereas it's very rare for CPUs to include more than a couple dozen processors). I experimented a little bit this week with the number of individual jobs into which I was splitting the task of drawing a frame. I'm currently using 65536 workgroups of 32 threads each (up from 1024 workgroups of 32 threads each); the program now seems to be taking less than 2 milliseconds to draw each frame fairly consistently (with occasional random single-frame spikes). Unfortunately, I don't know why splitting the workload this aggressively seems to be making it so much faster. lister.c also still seems to slow down while images are transitioning out; I don't know why this is either.

I mentioned in the September 15th log that lister.c currently has a bug in which it very occasionally crashes when trying to close. I now believe I fully understand the nature of this bug, and I believe it's fixed. I am no longer aware of any bugs anywhere in the program.

Having run out of ideas about further speeding up lister.c, I believe that now is a good time for me to learn how to use the Vulkan graphics pipeline. Learning how to draw even a simple triangle in Vulkan is a notoriously frustrating and time-consuming process (according to vulkan-tutorial.com even the most stripped-back possible program can require around nine hundred lines of code). I have a great deal of confidence in my ability to learn new things efficiently; even so, I might not be able to provide a lot of tangible results in the near-term future during this process. (Here's a picture of what a "my first triangle" program often ends up looking like, courtesy of vulkan-tutorial.com:)

As always, thank you for reading.

Weekly Summary Sep. 30 - Oct. 6

I changed how lister.c manages images this week. Although I haven't tested every possible image type, lister should now be compatible with most JPEG, PNG, BMP, TGA, PSD, HDR, PNM and PIC images; in theory it should also work with non-animated GIF files as well.

Unfortunately, I didn't make as much progress on lister.c as I would've liked to this week; I ended up spending most of the week working on this website. Putting it online probably took longer than strictly necessary but I wanted to make absolutely sure I got it right. I now believe it's built on a very, very steady foundation and I don't anticipate it occupying too much of my attention moving forward.

As always, thank you for reading.

Weekly Summary Sep. 23 - Sep. 29

I continued making additions to lister.c this week. The program now displays scaled-down versions of images which are too big for the screen, making it possible to compare pretty much any set of images you give it. Here's how it currently looks:

The file names are no longer hard-coded into the program, meaning you can now give it any directory you want (including from an external drive) and it will sort all of the images in that directory. Here's an example of how the output looks:

(I apologize for the inaccuracy of these rankings. Unfortunately, I compiled them before Week 4. Damn you, Steelers.)

lister.c is capable of processing an ever-increasing number of image formats. It can now handle all ".jpg" and ".jpeg" files as well as an overwhelming majority of ".png" files, including partially transparent ".png" files as demonstrated here.

As you can see, the window size is no longer arbitrary; it now occupies the full area of the screen. There's no toolbar; instead, the user can minimize the screen by pressing the spacebar. You can select an image by either clicking on it, by pressing the right or left arrow keys, or by pressing the 'A' (for left) and 'D' (for right) keys. As Vulkan and Wayland are the technologies I'm interested in learning at this time, naturally this program only works within Wayland compositors and with very specific Vulkan setups. I don't currently intend to change this.

lister.c uses Vulkan compute shaders to draw each individual frame. Unfortunately, increasing the size of the frame has slowed down the speed at which Vulkan draws them; it's currently peaking at between 4 and 5 milliseconds per frame in the worst case. I'm likely gonna spend some time experimenting with this because in theory Vulkan can make this program can run very, very fast. I'm also likely going to make it possible to save partially-completed lists to be reloaded at a later time. At this time, I don't think I'm likely to continue making further improvements to lister.c beyond these two. I'd now like to learn more about the Vulkan graphics pipeline so that I can continue challenging myself.

I continued researching computer networking this week and am inching ever-closer to posting these summaries online. I spent some time familiarizing myself with Apache and I now have an HTTP server running on my computer (although this server isn't connected to the Internet yet). Unfortunately, I stopped before formating the page and it looks extremely dry at this time, but I'm already very familiar with CSS and I'm not expecting the formatting process to take too long.

As always, thank you for reading.

Weekly Summary Sep. 16 - Sep. 22

I finished the first fully-working version of lister.c this week. At the moment, its functionality is relatively primitive, prompting the user to make a series of comparisons and then printing the resulting sorted list (made with a very small number of necessary comparisons thanks to the merge-insertion sort algorithm) to the terminal as follows:

While fully working, there's still a great deal of room for improvement over the near future:

• Right now, the size of the window is completely arbitrary and hard-coded into the program. Lister doesn't do any image scaling, so if the image is too big for the window then the user will only see a fraction of it.
• At this time, the program only processes JPEG-formatted images. It doesn't process PNG, SVG, or other completely standard image formats. It also only currently processes images in the locations "jpg/1.jpg", "jpg/2.jpg", and so on; the number of images it sorts is easily-changable but hard-coded into the program.
• There's significant room for improvement in how the results are displayed to the user. I'm thinking about adding some sort of end screen with a snapshot of the images in their sorted order. I'm also thinking about adding some sort of mechanism to display the user's current progress through a sort.
• There's no "back" button and no way to resume a previously-started sort.

Of course, the biggest shortcoming of this program remains the fact that it's only designed to work on my specific operating system. If the user's system isn't running a Wayland compositor specifically on an Nvidia GPU, the program almost certainly won't work. This portability problem, however, likely constitutes a relatively long-term time investment, and for the time being I'd prefer to focus on continuing to learn how to use Vulkan (although there is a chance that I take interest in developing an Android version of lister.c in the medium-term future).

Lister actually only occupied a small percentage of my total time on project this week. I've developed an interest in learning more about computer networking, a topic which I have an extremely limited grasp on at this time. At some point I'd like to buy server space onto which I can post these summaries, but I'd like to strengthen my technical fundamentals before making such an investment. The goal is to get to the point where I can simply rent out a server and run an HTTP processor on it which delivers these journals through the Internet as a formatted HTML file. Before I do so, however, I'd like to dry-run it by using my laptop as a network access point to which I can connect from my phone. I've already made some progress on this front, creating a local Wi-Fi network that appears in my phone's connection menu:

(Unfortunately, my phone doesn't actually connect to this network yet, stalling indefinitely when I try:)

I already have some experience with both front-end and back-end web development and am not anticipating too many complications on these fronts. However, I'm almost entirely new to the data link layer. Although I might work a little bit on lister.c on the side, I'm expecting research into this topic to take up a good deal of my time on project moving forward.

As always, thank you for reading.

Weekly Summary Sep. 9 - Sep. 15

I made two new Wayland programs this week, twojpg.c and lister.c. The fundamental idea is to keep learning how to make simple (at this point) but extremely fast GPU-accelerated applications using only Wayland protocol code and the Vulkan graphics API. The idea behind twojpg.c was to figure out how to get multiple images moving on the screen at the same time. lister.c is more ambitious: the idea is to basically use what I've learned from twojpg to build an application which allows the user to sort an arbitrarily large set of arbitrary images. The concept is to show the user two images at a time and to prompt them to pick their favorite of the two, and to keep doing so until we've ran enough comparisons to completely sort the list. I'm gonna use a sorting algorithm called merge-insertion sort to try to minimize the number of comparisons that the user needs to do for any given list.

Here's what they both look like so far:

So those colored boxes are actually imported JPEG images that I generated with genjpg.c. Although these specific images are pretty boring, they could actually theoretically be pretty much any image you can give it. For example, let me just run it with a dog picture as one of the inputs:

I haven't done anything with scaling yet so the images still have to have relatively specific dimensions in order to fit into the window. Normal-sized images typically take up more than the entire screen. At some point I'm probably gonna fix this but for the time being I'm more interested in getting the underlying functionality working.

The way twojpg works is that it uses three images and displays two at a time. When you click on either of the images, it starts sliding out to the side, and then once it's completely slid out, the third image starts sliding in to take its place. Normally I have the animation set to happen over 20 frames, relatively fast if your screen is running at 240 FPS, but I've tested it and everything still works fine when you click on an image mid-transition or make two images transition at the same time. This program runs indefinitely until the user presses the ESC key (or closes the program from the outside).

I spent some time reworking the multithreading structure for lister.c this week so I'm not super far along with it yet but you can already start to see the idea coming together. I've usually been testing it with ten images generated by genjpg although that number can change arbitrarily. What it does so far is show the user a pair of images; once the user clicks on either of the images, they both transition out and a new pair of images takes its place. The program exits once it gets through every pair; the user can also end the program at any time by pressing the ESC key.

Right now I'm working on fixing a bug in lister.c where it sometimes doesn't close properly after the last comparison. Unfortunately this bug is relatively rare and not super easy to replicate, so what I'm probably gonna do is just improve some of my multithreading code and hope for the best in the short term, and then circle back to it once I have more information about it if this doesn't end up fixing it. After that, I have the feeling that finishing the first working version of lister.c won't actually take that much longer. The plan is to have a minimally-complete lister.c simply print the custom-sorted list of images to the command line terminal although I might want to play around with something more user-friendly once I have that done.

Long-term, I'm currently leaning towards making a graphics-pipeline version of window.c to familiarize myself with the Vulkan graphics pipeline once I'm done with lister.c. However, I'm also tempted to look into the Android NDK to see if I can learn how to get lister working as an Android application. I also might try to spend some time trying to figure out why lister (and a lot of my other code for that matter) isn't working on KDE Plasma (which is peculiar as it works more-or-less seamlessly on Weston, GNOME, and even on Plasma itself over dGPU), although I don't expect this to take too long.

As always, thank you for reading.