Volume 22, Issue 4
Case Study
Program Merge: What's Deep Learning Got to Do with It?
A discussion with Shuvendu Lahiri, Alexey Svyatkovskiy, Christian Bird, Erik Meijer and Terry Coatta
If you regularly work with open-source code or produce software for a large organization, you're already familiar with many of the challenges posed by collaborative programming at scale. Some of the most vexing of these tend to surface as a consequence of the many independent alterations inevitably made to code, which, unsurprisingly, can lead to updates that don't synchronize. Difficult merges are nothing new, of course, but the scale of the problem has gotten much worse. This is what led a group of researchers at MSR (Microsoft Research) to take on the task of complicated merges as a grand program-repair challenge, one they believed might be addressed at least in part by machine learning.
AI,
Case Studies
Research for Practice
Deterministic Record-and-Replay
Peter Alvaro, Andrew Quinn
Zeroing in only on the nondeterministic actions of the process
This column describes three recent research advances related to deterministic record-and-replay, with the goal of showing both classical use cases and emerging use cases. A growing number of systems use a weaker form of deterministic record-and-replay. Essentially, these systems exploit the determinism that exists across many program executions but intentionally allow some nondeterminism for performance reasons. This trend is exemplified in GPUReplay in particular, but also in systems such as ShortCut and Dora.
Debugging,
Research for Practice
Bridging the Moat
Test Accounts: A Hidden Risk
You may decide the risks are acceptable. But, if not, here are some rules for avoiding them.
A test account that's shared among many can be used by anyone who happens to have the password. This leaves a trail of poorly managed or unmanaged accounts that only increases your attack surface. A test account could be a treasure trove of information, even revealing information about internal system details. If you really need to take this approach, give your developers their own test accounts and then educate them about the risks of misusing these accounts. Also, if you can periodically expire these accounts, all the better.
Bridging the Moat,
Security
Kode Vicious
Unwanted Surprises
When that joke of an API is on you
There is the higher-order question of whether loosely typed languages with coercion are really a good idea in the first place. If you don't know what you're operating on, or what the expected output range might be, then maybe you ought not to be operating on that data in the first place. But now these languages have gotten into the wild and we'll never be able to hunt them down and kill them soon enough for my liking, or for the greater good.
Development,
Kode Vicious
GPTs and Hallucination
Jim Waldo, Soline Boussard
Why do large language models hallucinate?
The findings in this experiment support the hypothesis that GPTs based on LLMs perform well on prompts that are more popular and have reached a general consensus yet struggle on controversial topics or topics with limited data. The variability in the applications's responses underscores that the models depend on the quantity and quality of their training data, paralleling the system of crowdsourcing that relies on diverse and credible contributions. Thus, while GPTs can serve as useful tools for many mundane tasks, their engagement with obscure and polarized topics should be interpreted with caution. LLMs' reliance on probabilistic models to produce statements about the world ties their accuracy closely to the breadth and quality of the data they're given.
AI,
Privacy and Rights
Confidential Computing Proofs
Mark Russinovich, Cédric Fournet, Greg Zaverucha, Josh Benaloh, Brandon Murdoch, Manuel Costa
An alternative to cryptographic zero-knowledge
Proofs are powerful tools for integrity and privacy, enabling the verifier to delegate a computation and still verify its correct execution, and enabling the prover to keep the details of the computation private. Both CCP and ZKP can achieve soundness and zero-knowledge but with important differences. CCP relies on hardware trust assumptions, which yield high performance and additional confidentiality protection for the prover but may be unacceptable for some applications. CCP is also often easier to use, notably with existing code, whereas ZKP comes with a large prover overhead that may be unpractical for some applications.
Privacy and Rights,
Security
Assessing IT Project Success: Perception vs. Reality
João Varajão, António Trigo
We would not be in the digital age if it were not for the recurrent success of IT projects.
This study has significant implications for practice, research, and education by providing new insights into IT project success. It expands the body of knowledge on project management by reporting project success (and not exclusively project management success), grounded in several objective criteria such as deliverables usage by the client in the post-project stage, hiring of project-related support/maintenance services by the client, contracting of new projects by the client, and vendor recommendation by the client to potential clients. Researchers can find a set of criteria they can use when studying and reporting the success of IT projects, thus expanding the current perspective on evaluation and contributing to more accurate conclusions. For practitioners, this study provides a rich set of criteria that can be used for evaluating their projects, as well as strong evidence of the importance of considering not only project execution, but also post-project outcomes and impacts in the evaluation.
Business and Management,
Education
Questioning the Criteria for Evaluating Non-cryptographic Hash Functions
Catherine Hayes, David Malone
Maybe we need to think more about non-cryptographic hash functions.
Although cryptographic and non-cryptographic hash functions are everywhere, there seems to be a gap in how they are designed. Lots of criteria exist for cryptographic hashes motivated by various security requirements, but on the non-cryptographic side there is a certain amount of folklore that, despite the long history of hash functions, has not been fully explored. While targeting a uniform distribution makes a lot of sense for real-world datasets, it can be a challenge when confronted by a dataset with particular patterns.
Development
Volume 22, Issue 3
Virtual Machinations: Using Large Language Models as Neural Computers
Erik Meijer
LLMs can function not only as databases, but also as dynamic, end-user programmable neural computers.
We explore how Large Language Models (LLMs) can function not just as databases, but as dynamic, end-user programmable neural computers. The native programming language for this neural computer is a Logic Programming-inspired declarative language that formalizes and externalizes the chain-of-thought reasoning as it might happen inside a large language model.
AI,
Development,
Virtualization
Toward Effective AI Support for Developers
Mansi Khemka and Brian Houck
A survey of desires and concerns
The journey of integrating AI into the daily lives of software engineers is not without its challenges. Yet, it promises a transformative shift in how developers can translate their creative visions into tangible solutions. As we have seen, AI tools such as GitHub Copilot are already reshaping the code-writing experience, enabling developers to be more productive and to spend more time on creative and complex tasks. The skepticism around AI, from concerns about job security to its real-world efficacy, underscores the need for a balanced approach that prioritizes transparency, education, and ethical considerations. With these efforts, AI has the potential not only to alleviate the burdens of mundane tasks, but also to unlock new horizons of innovation and growth.
AI,
Development
You Don't Know Jack about Bandwidth
David Collier-Brown
If you're an ISP and all your customers hate you, take heart. This is now a solvable problem.
Bandwidth probably isn't the problem when your employees or customers say they have terrible Internet performance. Once they have something in the range of 50 to 100 Mbps, the problem is latency, how long it takes for the ISP's routers to process their traffic. If you're an ISP and all your customers hate you, take heart. This is now a solvable problem, thanks to a dedicated band of individuals who hunted it down, killed it, and then proved out their solution in home routers.
Networks,
Performance
Transactions and Serverless are Made for Each Other
Qian Li, Peter Kraft
If serverless platforms could wrap functions in database transactions, they would be a good fit for database-backed applications.
Database-backed applications are an exciting new frontier for serverless computation. By tightly integrating application execution and data management, a transactional serverless platform enables many new features not possible in either existing serverless platforms or server-based deployments.
Databases,
Distributed Computing
The Soft Side of Software
Working Models for Tackling Tech Debt
Kate Matsudaira
Understand the options to tailor an approach that suits your needs
Remember that not all debt is bad, and sometimes, in fact, strategic tech debt can even be used as a valuable tool to achieve certain business goals?just as financial debt can be taken on to obtain capital that can be invested in other profitable ventures. For example, taking a shortcut to get a product to market quickly could prove to be a wise decision if it allows the company to learn from customer feedback and then iterate accordingly on the product. But like barnacles on a ship, too much tech debt can slow you down, so be vigilant about managing it.
Business/Management
The Soft Side of Software
Kode Vicious
Repeat, Reproduce, Replicate
The pressure to publish versus the will to defend scientific claims
Unless a result relies on a specific hardware trick, such as a proprietary accelerator or modified instruction set, it is possible to reproduce the results of one group by a different one. Unlike the physicists we don't have to build a second Hadron Collider to verify the result of the first. We have millions of similar, and sometimes identical, devices, on which to reproduce our results. All that is required is the will to do so.
Education,
Kode Vicious
The Bikeshed
The Expense of Unprotected Free Software
It's high time FOSS maintainers got a bit of appreciation
Poul-Henning Kamp
Until the big guns manage to sort things out, we're just going to need to take care of things however we can. The best we can hope for, of course, is to convince companies, institutions, and governments that it would be a really good idea to cut monthly checks for those people who maintain the software that these organizations absolutely depend upon.
The Bikeshed,
Open Source
Volume 22, Issue 2
Drill Bits
Zero Tolerance for Bias
Terence Kelly
From gambling to military conscription, randomization makes crucial real-world decisions. With blood and treasure at stake, fairness is not negotiable. Unfortunately, bad advice and biased methods abound. We'll learn how to navigate around misinformation, develop sound methods, and compile checklists for design and code reviews.
Drill Bits,
Code,
Development,
Performance
Kode Vicious
Structuring Success
The problem with software structure is people don't really learn it until they really need it.
Dear KV, In teaching an algorithms course this semester, I discovered my students had received very little instruction about how to divide their code into functions. So, I spent a weekend trolling various programming handbooks and discovered most of them are silent on this topic. I ended up writing a quick handbook to help my students, but was struck more by the advice gap. We just don't give people guidance!
Development,
Education,
Kode Vicious
Trustworthy AI using Confidential Federated Learning
Jinnan Guo, Peter Pietzuch, Andrew Paverd, Kapil Vaswani
Federated learning and confidential computing are not competing technologies.
The principles of security, privacy, accountability, transparency, and fairness are the cornerstones of modern AI regulations. Classic FL was designed with a strong emphasis on security and privacy, at the cost of transparency and accountability. CFL addresses this gap with a careful combination of FL with TEEs and commitments. In addition, CFL brings other desirable security properties, such as code-based access control, model confidentiality, and protection of models during inference. Recent advances in confidential computing such as confidential containers and confidential GPUs mean that existing FL frameworks can be extended seamlessly to support CFL with low overheads. For these reasons, CFL is likely to become the default mode for deploying FL workloads.
AI,
Security
Confidential Computing or Cryptographic Computing?
Raluca Ada Popa
Tradeoffs between cryptography and hardware enclaves
Secure computation via MPC/homomorphic encryption versus hardware enclaves presents tradeoffs involving deployment, security, and performance. Regarding performance, it matters a lot which workload you have in mind. For simple workloads such as simple summations, low-degree polynomials, or simple machine-learning tasks, both approaches can be ready to use in practice, but for rich computations such as complex SQL analytics or training large machine-learning models, only the hardware enclave approach is at this moment practical enough for many real-world deployment scenarios.
Hardware,
Security
Confidential Container Groups
Matthew A. Johnson, Stavros Volos, Ken Gordon, Sean T. Allen, Christoph M. Wintersteiger, Sylvan Clebsch, John Starks, Manuel Costa
Implementing confidential computing on Azure container instances
The experiments presented here demonstrate that Parma, the architecture that drives confidential containers on Azure container instances, adds less than one percent additional performance overhead beyond that added by the underlying TEE (i.e., AMD SEV-SNP). Importantly, Parma ensures a security invariant over all reachable states of the container group rooted in the attestation report. This allows external third parties to communicate securely (via remote attestation) with containers, enabling a wide range of containerized workflows that require confidential access to secure data. Companies obtain the advantages of running their most confidential workflows in the cloud without having to compromise on their security requirements. Tenants gain flexibility, efficiency, and reliability; CSPs get more business; and users can trust that their data is private, confidential, and secure.
Architecture,
Security
Operations and Life:
Make Two Trips
Thomas A. Limoncelli
Larry David's New Year's resolution works for IT too.
Whether your project is as simple as carrying groceries into the house or as complex as a multiyear engineering project, "make two trips" can simplify the project, reduce the chance of error, improve the probability of success, and lead to easier explanations.
Business and Management,
Development,
Operations and Life,
Systems Administration
Elevating Security with Arm CCA
Charles Garcia-Tobin, Mark Knight
Attestation and verification are integral to adopting confidential computing.
Confidential computing has great potential to improve the security of general-purpose computing platforms by taking supervisory systems out of the TCB, thereby reducing the size of the TCB, the attack surface, and the attack vectors that security architects must consider. Confidential computing requires innovations in platform hardware and software, but these have the potential to enable greater trust in computing, especially on devices that are owned or controlled by third parties. Early consumers of confidential computing will need to make their own decisions about the platforms they choose to trust. As confidential computing becomes mainstream, however, it's possible that certifiers and regulators will share this burden, enabling customers to make informed choices without having to undertake their own evaluations.
Privacy and Rights,
Security