This is the fourteenth video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

Appeals to prior technological revolutions are often used to assuage fears that AI will lead to widespread unemployment. Yes, you may lose your job because of AI, but eventually you'll get a different job in a new field created by the resulting technological progress - or so the argument goes.

In this episode of Wading Through AI, Demetri and Casey discuss whether this argument holds water, or if it's just wishful thinking unjustified by direct historical analogy.

Subscribe now

As always, the art for the thumbnail, podcast cover, and banner of Wading Through AI were all hand-drawn by Anna Rettberg in her signature style. We do not currently use generative AI here at Molly Rocket.

This is the thirteenth video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

AI companies seem fixated on scaling. From massive marketing budgets to supply-chain-altering datacenter plans, the drive to scale appears to be more urgent and aggressive than in any tech boom that came before.

In this episode of Wading Through AI, Demetri walks Casey through the list of factors that make scaling a necessity for companies competing to be the AI providers of the future.

Subscribe now

As always, the art for the thumbnail, podcast cover, and banner of Wading Through AI were all hand-drawn by Anna Rettberg in her signature style. We do not currently use generative AI here at Molly Rocket.

Last month, I posted Let’s Answer Ray’s Question! - a video where we looked at the memory behavior of two different compilations of the same program. I ended that video by saying that if you could read assembly language (like you can if you did Part I of the Performance Aware Programming course!), you could go much further and determine even more specifics.

In this video, we do exactly that. Using disassembly, memory view, and heap debugging, we zero in on exactly who is setting the magic eight bytes we saw next to Ray’s allocations in the first video - and we figure the convoluted way those bytes are encoded.

If you’d like to be emailed when the third and final video is posted, you can subscribe your email address to either our free or paid tiers using the button below.

Subscribe now

With previously open source projects going closed, and artists increasingly concerned about posting their work online, is AI reversing the internet’s long-established trend toward openness?

In this sixth episode of Wading Through AI, Demetri and I consider the possibility that AI could substantially diminish, or even eventually end, the norms of the open internet.

Subscribe now

As always, the art for the thumbnail, podcast cover, and banner of Wading Through AI were all hand-drawn by Anna Rettberg in her signature style. We do not currently use generative AI here at Molly Rocket.

For those wondering what the name of the audio API was that made me go “I can’t believe I am forgetting the acronym” in the middle of this video, it was ASIO.

The questions addressed in this video are:

• [00:02] “The D language actually has this because of Uniform Function Call Syntax. You could have a struct like … and have the rest of the code be none the wiser (even code like foo.x = 7). I think C# has stuff like this too. Is this what you’d be hoping for? If so, I’m surprised - I’d have guessed you usually don’t prefer implicit magic like this happening on things that don’t look like function calls.”

• [04:37] “Could someone remind me what the ‘warm’ criteria was?”

• [06:00] “I have a question about managing intrusive thoughts while writing code. When thoughts come up about edge cases, performance, reliability, or decisions that may need to be revisited later, how do you decide whether to address them immediately or set them aside? I’m finding it difficult to iterate quickly because I often get pulled into these concerns before I’ve built a working v0. I’ve tried writing TODOs in the code and keeping separate notes, but neither approach has worked well for me.”

• [15:47] “I now have a question about wasapi (I believe you now use wasapi but if I’m wrong then you can ignore this question). I’m writing the sound system for my codebase and had my wav parser ready, mixer chugging along, sat back to enjoy a song but then found out that changing the system volume causes these pops.... I know that changing the volume of a sound wave causes the discontinuity which manifests itself as audio popping but in this case where it’s not an internal volume I have control over I don’t know how to fix these. It seems to me as if windows just immediately applied the new volume and my futile attempt of trying to bypass windows’ volume logic didn’t work since they seem to first clamp my samples and then multiply by the volume... I tried some wasapi gist examples and they all had this same issue. Have you had this problem and fixed it in your own code and could you share your knowledge if you have?”

• [25:07] “I formed my question about heaps slightly misleading. It wasn’t about a memory heap, but a data structure, like in the heap sort. In the video together with Primeagen regarding code interviews you commented that heaps are quite slow. So I was wondering, what should you use instead, when the need arises?”

• [33:27] “I’m working on a project that uses a cortex-m7, a fairly fast 32-bit embedded core. It has some fancy pipeline features of big boy desktop chips, like a branch predictor, and the ability to dual issue many instructions, but doesn’t have a RAT (it’s not “out-of-order” at least as far as I know). Do I understand correctly that in CPUs without a RAT, any two adjacent instructions that use the same register are effectively serially dependent? Is this how it used to work in the desktop world back in the day?”

• [40:53] “This question relates to Part 5: Dependency Chain Stalls and In-order Interleaving. From these results, I would expect 12-way interleaving to maintain peak throughput regardless of chain length (4 FMAs/cycle * 3 cycles = 12 FMAs in flight needed to cover the latency). However, that’s not what I observe. With 12-way interleaving, throughput decreases from ~3.6 to ~3.0 FMAs/cycle as the chain length grows. I need a minimum of 16-way interleaving for throughput to stay flat at ~3.8 FMAs/cycle. My conclusion is that the CPU can’t sustain the 3-cycle latency when the FMA execution ports are saturated, resulting in an effective latency of 4 cycles under load. This would explain why I need 4 * 4 = 16 chains to obtain a flat throughput curve. I think you’ve hinted at this in some of your videos. Would you kindly explain why this happens? Thanks!”

Read more

Suppose you want to pick a random element from a set that is difficult to iterate. Perhaps the candidate elements are embedded in a spatial hierarchy, and only determined by a complex predicate.

One obvious way to solve this problem is to first build a “filtered” array of all the candidate elements, then pick a random index into that array. This always works, and is fair (assuming the RNG is fair) — but it requires reserving space for a potentially-large array, and uses up bandwidth and cache space each time a random selection is made.

A non-obvious-but-common alternative is to use a statistical method where random numbers are chosen during the initial iteration to continuously “retain or replace” a running selection. The random numbers must be carefully constructed, and typically require a modulus on each step. Proof of the fairness of the method is not obvious, and requires knowledge of the relevant underlying math.

In short, while it works, and is likely more efficient than the array method, it has drawbacks.

Inspired by the way you might write such a query in SQL, Marc “Mahk” LeBlanc shared with me a new arrayless picking method that has none of these drawbacks. It’s very similar to the statistical methods version, but without any of the drawbacks.

In this video, Mahk presents the algorithm in a series of steps, starting from a “worst” random selector, and ending with his new one.

Subscribe now

As people integrate more and more AI into their daily work, will the quality of the work they produce suffer? Will their own skills degrade as a result of relying on AI? Will there be negative psychological effects in the shift from doing work directly to doing it indirectly through AI prompting?

In this episode, Demetri and I explore these issues as we consider the broad question many people are asking themselves: “Will AI make me worse?”

Subscribe now

As always, the art for the thumbnail, podcast cover, and banner of Wading Through AI were all hand-drawn by Anna Rettberg in her signature style. We do not currently use generative AI here at Molly Rocket.

At last year’s inaugural Better Software Conference, I gave a presentation called The Big OOPs. One of the central figures in the presentation was programmer-designer Marc “Mahk” LeBlanc, whom I credited with the key phrase, “a compile-time hierarchy that matches the domain model”.

That quote, and the chronology of Looking Glass’s entity systems, both came from an extended interview I did with Mahk as part of my research for the talk. At the end of that interview, Mahk and I both had the same thought: “We should have recorded this!” There was so much Looking Glass history, nicely focused specifically on the evolution of their entity systems, that it seemed a shame nobody would get to hear it.

So this year, we did the only logical thing: we both pretended we had never had the conversation, and conducted the whole interview again. This time, we recorded it.

And here it is! The full two-and-a-half-hour interview, covering everything from Mahk’s first encounter with Looking Glass1 during Ultima Underworld, up through his work on the core engine architecture for Thief: The Dark Project.

Bonus Unboxing

Because we could not find any corroboration posted on the internet, Mahk also unboxed a still-shrink-wrapped, original boxed copy of Flight Unlimited so we could verify it did, in fact, ship with a batch file that restarted the game to fix C++-induced memory fragmentation:

After installing the game directly from the original CD, Mahk found the following batch file used to run the game:

@echo off
set dos4g=quiet
Y:\cdrom\flight.exe %1 %2 %3 %4 %5 %6 %7 %8 %9
:wacky
if errorlevel 44 goto done
if errorlevel 43 goto refuel
if errorlevel 42 goto credit
goto done
:credit
Y:\cdrom\credits.exe
Y:\cdrom\flight.exe +fromcredits %1 %2 %3 %4 %5 %6 %7 %8 %9
goto wacky
:refuel
Y:\cdrom\flight.exe +refuel %1 %2 %3 %4 %5 %6 %7 %8 %9
goto wacky
:done

To quote Mahk, “If anything marks this as a genuine LG2 product, it’s the use of ‘wacky’ as a goto label.”

Mahk Links

I hope you enjoy the interview! If you’d like to check out what Mahk’s been up to since the Looking Glass days, you can find him at the following links:

• Mahk’s YouTube: https://www.youtube.com/@algorithmancyTube

• Mahk’s Twitch: https://www.twitch.tv/algorithmancy/videos

• Mahk’s game design website: https://8kindsoffun.com

Until next time, have fun programming everyone, and I (and Mahk!) will see you on the internet.

— Casey

This is the first in a series of Big OOPs supplemental materials that I will be posting over the next several weeks. In addition to this interview with Mahk, there will be paper reviews of each of the major tranches that I used to reconstruct the history presented in the talk. If you’d like to get those delivered to your inbox when they’re published, you can take a look at our subscription options here:

Subscribe now

This is the twelfth video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

The questions addressed in this video are:

• [0:00:03] “I would like to learn/understand more about performance oriented architecture design as most of the performance resources I see are on pinhole omptimizations, do you have specific examples and general guidelines for that?”

• [0:05:10] “What are the architectural considerations to allow the compiler to SIMDify your code?”

• [0:08:13] “You once mentioned that heaps are slow in real life. Could you elaborate on why and which algorithms may be used instead in appropriate situations?”

• [0:11:09] “I saw some advice outside of this course which was to try multiple overlapped reads at once and bypassing the file cache with FILE_FLAG_NO_BUFFERING. Trying this with 32 overlapped reads and reading the whole file into one big buffer (no chunking), I’m now reading at 6 GB/s. This is in line with my SSD read speed (7GB/s allegedly), but not all the numbers are making sense to me. I have two questions:

  How can bypassing the file cache and reading from SSD possibly be faster than reading from the file cache in memory? My main memory is 64GB so I imagine my whole file fits in the cache.

  Regardless of overlapped reads or bypassing the file cache, if I read the whole file into one big buffer, shouldn’t I be paying the cost of page faults regardless? I’d expect page faults and cache inefficiency would limit me to the original 2.5GB/s.”

• [0:19:32] “With block interleaving, we are doing extra loads/stores, how much more expensive is that in terms of power consumption? does it make sense to do the in order interleaving on some kind of mobile device specifically for battery life or to help maintain higher boost clocks via lower temperatures?”

• [0:21:54] “At my job we use Unreal and I find myself writing very defensive code. There is so much “if object exists, do thing, else, log an error and return” across the code base, it’s starting to feel like a waste of time. Especially since in a large number of cases, an early return still leaves the game in an unplayable state. The problem isn’t fixed; only moved elsewhere.

  Do you have any thoughts on when one should do this kind of granular error checking, when one should assert and when it’s okay to just let it crash?

  Is there a better way or do I just have to keep adding all this noise to my code?”

• [0:25:40] “Is it fine to expose internals of a struct together with “helper functions” to calculate derived data? Should I not worry about exposing internals? Any pointers to where I can learn more about library design in C?”

• [0:33:31] “In one QA you said that JIT and compile time are not mutually exclusive, but why I never seen some software written in a “JIT-less” language and use a JIT library to fill the compile time gap performance?”

• [0:36:40] “Something that its still not clear to me is: how do I choose a RAM frequency when buying a PC?”

• [0:40:09] “You said somewhere that you are not a fan o End To End (E2E) tests, why is that?”

• [0:45:03] “I’m curious about what to do when a procedure has only one use in the code? How to conciliate this with semantic compression? Assuming that you can only compress something if it was written at least twice. Because sometimes, even though its used once, it makes easier to read.”

• [0:47:26] “I was wondering: is a relational DB also the 35 year mistake? Since its a statically typed hierarchy that models the domain. My gut feeling is that its not, but it fits very well into the description.”

• [0:49:32] “In this article the author cites some research papers about how the SOLID principles improve software: https://florian-kraemer.net/software-architecture/2025/02/24/Are-the-SOLID-Principles-problematic.html and related to that, the book Agile software development: Principles, Patterns and Practices by Bob Martin, also tries to give a measurement ‘framework’ for SOLID. Any thoughts on this?”

• [0:53:53] “In OO code is common to use objects with private data and some methods and constructor to keep the internal data always in a valid state, but in a procedural way, like C, you can misuse a struct. How do you deal with this in C or in procedural way in general?”

• [0:59:14] “Do you think that is worth creating content like the “OOPs” talk, “Clean Code Horrible Performance”, “Where does bad software come from?” (SOLID) where it tries to persuade the viewer that the “best practice” he was taught is bad? Because even though you are very sharp in your reasoning, your content still get misinterpreted for several reasons. I was thinking of doing something similar myself, grouping everything I learned from the “Handmade way” to present a better way to program, but I’m not sure if its worth it and I don’t have decades of experience so I’m afraid my argumentation would not be good enough.”

• [1:05:04] “Related to the question above, how do I convince my coworkers that they are doing bad practices?”

• [1:06:35] “How exactly do I apply WARMED at work?”

• [1:08:59] “In the end of your IMGUI talk, you said that was excited to see what would be improved in this approach in the next years. So, after 20 years, which improvements you saw?”

• [1:11:31] “Why compilers don’t have a #pragma DONT_REMOVE, #pragma END_DONT_REMOVE to make our lives easier instead of writing assembly code to do fake reads and writes for measuring performance for a block of code?”

• [1:13:13] “I’m curious on what a web dev handmade hero would look like, because in the game version is possible to ship a commercial game without using any libraries, however I don’t think that is possible with web, since you need some libraries to deal with security/encryption like HTTPS. So how would you approach this if you were to make a handmade hero for web? Like showing how to create a substack clone, but better :)”

• [1:14:59] “Knowing the specs of the Lion Cove P-Core of that CPU with 3x load units/ports it makes sense that there is no additional speedup after Read_x3. What puzzles me a bit/makes me curious though is the fact that Read_x3 is consistently the fastest. I tried it numerous times, locked the program to the same single P-core, built dedicated executables to make sure it has nothing to do with some code alignment etc. but Read_x3 is always the fastest with some ~2% edge over Read_x4. Of course 2% is not a huge difference, but I would have assumed that Read_x3 and Read_x4 would converge more or less or if at all that Read_x4 would be slightly faster due to less loop overhead in total.”

• [1:19:22] “Now that you’re mainly on Linux, have you been using any of the profiling tools that Linux provides in <linux/perf_event.h> and if you do, what are your thoughts on them?” / “In a recent standup I heard how you are now fully switched to Linux and never going back to Windows. I would like to know, what are you now then using for debugging on Linux? As raddbg is not yet ready, though getting closer…”

Read more

A few days ago, Ramon “Ray” Santamaria, author of the popular library Raylib, posted the following code snippet and question on X:

As you can see from the screenshot, GCC and MSVC x64 appear to create binaries that behave “correctly” - or at least not obviously wrong - whereas MSVC x86 appears to produce a binary that behaves “incorrectly”.

Although many people replied to Ray’s post by saying, “it’s undefined behavior” - which is true, because there’s a missing null terminator - that’s not a complete answer. That only says why the execution is allowed to be different, not how it actually differs.

I thought this was a great opportunity to show how we would investigate behavior like this to determine the root cause. I’ve prepared two videos (and some “outtakes” with esoterica) that show this process.

In this first video, I walk through the entire “source language” side of the behavior to show what is actually causing it. I will be posting a second video shortly for people who can read ASM which shows how to go one layer deeper and look at an even more precise answer.

If you’d like to be emailed when the second video is posted, you can subscribe your email address to either our free or paid tiers using the button below.

Subscribe now

This is the eleventh video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

The legal issues surrounding generative AI are something only the court system can decide. The ethical issues, on the other hand, are things that everyone must decide for themselves. Different people with different value systems will come to different conclusions.

In this episode, Demetri and I open the can of worms that is “AI ethics” and walk through a series of incremental steps people should think through when deciding for themselves if they think generative AI is ethical.

Subscribe now

As always, the art for the thumbnail, podcast cover, and banner of Wading Through AI were all hand-drawn by Anna Rettberg in her signature style. We do not currently use generative AI here at Molly Rocket.

The questions addressed in this video are:

• [00:04] “Quick question (for Casey or anyone with a suggestion!): I am neither a novice nor an experienced programmer, and would like to learn enough C++ to write some useful code, but I am struggling to find good material.”

• [02:18] “In the episode ‘In-order Interleaving’, you have a loop that handles multiple elements per loop iteration. When the list of elements isn’t a multiple of your ‘elements per loop’, we need something after the loop to handle the last few remaining elements. My question is: Is there a particular way to write this ‘residual handling’ part? Whenever I’ve had to do this in my own code it always felt a little awkward.”

• [13:25] “Hey Casey, what’s your take on Arm making their own chips now? Sincerely, an Arm engineer.”

• [17:05] “On the topic of growable arenas, concretely, how do you implement them with respect to the ‘layered’ architecture? To grow them, naturally you would need to mmap or VirtualAlloc more memory, but if you’re far removed from the platform layer and aren’t writing a program that is frame-based (for example, a compiler) and thus have no opportune time to go back to the platform layer and request more memory, what are your strategies? I can only come up with round-tripping back to the OS using a platform-layer API like ‘RequestMoreMemory()’, or something like this. For more context, this would be for the kind of program that also cannot place upper-bounds on memory usage (again, like in a compiler), but still does not want to be malloc’ing/free’ing excessively. I’m keen to hear how you would approach this problem?”

• [26:14] “Have you had occasion to use coroutines to write algorithms that must pause/resume? E.g., protocols in networking or software in a hardware simulator.”

Read more

When it comes to AI, people seem to be having very disparate experiences. Some people become enthusiastic, while others become frustrated, even when apparently attempting similar tasks.

In this episode of Wading Through AI, Demetri and I address the root causes of the feeling a lot of people - both pro- and anti-AI - are having that either they must be crazy, or everyone else must be, because their experiences seem to differ so dramatically from what they are hearing from others.

Subscribe now

As always, the art for the thumbnail, podcast cover, and banner of Wading Through AI were all hand-drawn by Anna Rettberg in her signature style. We do not currently use generative AI here at Molly Rocket.

This is the tenth video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

Technically, it was dated April 1st, so it may all have been a ruse! But April Fools joke or not, the vast majority of replies took it seriously.

While anyone can subjectively agree or disagree with Notch on the value of DLSS, there’s an underlying objective question regarding his use of the phrase “THE SAME HARDWARE”. Many replies took issue with this wording, claiming it was a clear sign Notch did not understand how DLSS worked.

I’m not so sure about that.

I think there is a good case to be made that - given what little NVIDIA tells us - Notch is not objectively wrong about NVIDIA hardware. I didn’t feel like a contorted X post would be the best way to elaborate on this, so I livestreamed my thoughts on Twitch, YouTube, and here on Substack.

The video is included above for those of you that missed the stream.

In this second episode of Wading Through AI, Demetri and I discuss the recent Anthropic blog post about “parallel Claudes” building a C Compiler, and use it as a case study for how to assess AI announcements.

Subscribe now

As always, the art for the thumbnail, podcast cover, and banner of Wading Through AI were all hand-drawn by Anna Rettberg in her signature style. We do not currently use generative AI here at Molly Rocket.