Frameworks

Vol. 23 No. 1 – January/February 2025

Frameworks

The Surprise of Multiple Dependency Graphs:
Dependency resolution is not deterministic.

It seems like it should be easy to avoid installing vulnerable open source software, but dependency graphs are surprisingly complex. At the time of writing, the latest version of the popular npm tool webpack has millions of potential dependency graphs depending on circumstances during its resolution. The exact graph chosen for a given package can depend on what other software is being built, what kind of system is building it, and even the state of the ecosystem on a given day. As a result, the developer and user of a package may end up with very different dependency graphs, which can lead to unexpected vulnerabilities.

by Josie Anugerah, Eve Martin-Jones

Fifty Years of Open Source Software Supply Chain Security:
For decades, software reuse was only a lofty goal. Now it's very real.

The xz attack seems to be the first major attack on the open source software supply chain. The event-stream attack was similar but not major, and Heartbleed and Log4j were vulnerabilities, not attacks. But the xz attack was discovered essentially by accident because it made sshd just a bit too slow at startup. Attacks, by their nature, try to remain hidden. What are the chances we would accidentally discover the very first major attack on the open source software supply chain in just a few weeks? Perhaps we were extremely lucky, or perhaps we have missed others.

by Russ Cox

String Matching at Scale:
A call for interdisciplinary collaboration and better-directed resources

String matching can't be that difficult. But what are we matching on? What is the intrinsic identity of a software component? Does it change when developers copy and paste the source code instead of fetching it from a package manager? Is every package-manager request fetching the same artifact from the same upstream repository mirror? Can we trust that the source code published along with the artifact is indeed what's built into the release executable? Is the tool chain kosher?

by Dennis Roellke

How to Evaluate AI that's Smarter than Us:
Exploring three strategies: functional correctness, AI-as-a-judge, and comparative evaluation

Evaluating AI models that surpass human expertise in the task at hand presents unique challenges. These challenges only grow as AI becomes more intelligent. However, the three effective strategies presented in this article exist to address these hurdles. The strategies are: Functional correctness: evaluating AI by how well it accomplishes its intended tasks; AI-as-a-judge: using AI instead of human experts to evaluate AI outputs; and Comparative evaluation: evaluating AI systems in relationship with each other instead of independently.

by Chip Huyen

From Function Frustrations to Framework Flexibility:
Fixing tool calls with indirection

The principle of indirection can be applied to introduce a paradigm shift: replacing direct value manipulation with symbolic reasoning using named variables. This simple yet powerful trick directly resolves inconsistencies in tool usage and enables parameterization and abstraction of interactions. The transformation of function calls into reusable and interpretable frameworks elevates tool calling into a neuro-symbolic reasoning framework. This approach unlocks new possibilities for structured interaction and dynamic AI systems.

by Erik Meijer

Analyzing Krazy Kode:
Accounting for the emotional state of the person who wrote that code

There actually are about six or seven emotions, or so I'm told. But the one state you should really try to avoid is confusion, which isn't actually an emotion but instead a state of mind. Code created by a confused mind shows itself in the randomness of naming, which is not handled by modern, fascist, programming languages like Go. Sure, you may have your names in the proper case and your spaces in the proper place, but you can still name a function PublicThingTwo() if you want to, and this is a sure sign of trouble.

by George Neville-Neil

A Clean Approach to Process Optimization:
What I learned from my dishwasher about automating processes

My soap-loading technique isn't revolutionary, but it does demonstrate a point about process design: You can eliminate delays in starting a process by front-loading tasks whenever possible. Front-loading changes when you do tasks but not their order. The process still involves a loop: load dishes, add soap, press start button, empty dishes repeat. You've only changed your mental model of where the loop starts.

by Thomas A. Limoncelli