TASM Notes, May 16th, 2024

Pre-Meeting Chatting

There are a bunch of upcoming events!

• Applications open for EAGx Toronto. August 16-18, InterContinental Toronto Center
• Online course: BlueDot Impact's AI safety fundamentals alignment course
  • Apply by June 02
  • Especially aimed at people considering a career in technical AI safety
• There's also a course based on this book, which you can find here

The News

GPT-4o has been released! It's not GPT-5. Still has roughly GPT4-level reasoning, but can go faster, better, cheaper. Which is... pretty awesome? At this point we de-rail by poking at our OpenAI apps and making gratuitous Her) comparisons for a few minutes. Nothing listed in the demos about music, even though I kinda want to see Fooming Shoggoths album #2.

Illya Sutskever and Jan Leike have left the superalignment team at OpenAI. This is sad and worrying.

There's an Ontario law regarding public sector use of AI. At the high level, if you're interacting with an AI in a public sector system, you need to be told that, and there must be human oversight of AI-made decisions, and deployers need to follow the OTAIF (there was an AIGS article written about this; link coming once it gets published).

The Carbon Emissions of Writing and Illustrating Are Lower for AI than for Humans

The Talk - Secure, Governable Chips

An Anecdote

John Deere and the Ukraine War: In February 2022, Russian forces took control of Melitopol, a Ukranian grain producing city? They shipped a bunch of gear and grain back to Russia, and then found that the tractors were remotely disabled by John Deere. I am very curious what Louis Rossmann's take on this would be.

This points to a potential use of "trusted computing" style systems in AI governance.

Legally, these are already in place; the US has export control surrounding AI tech (and GPUs in particular). The US, the Netherlands, Taiwan and South Korea are the main owners of the GPU supply chain right now, so it's possible to enforce this on US adversaries. The disadvantage here is that China's extensive civil-military fusion and use of shell entities help them evade export controls. Chip smuggling. The rules have workarounds too; cloud computing is still an option, and companies can stockpile current frontier cards in order to guard against the possibility of new or expanded export controls.

A More Surgical Approach

By ensuring chips are designed with certain safeguards in place, we can get more surgical about usage restriction than export controls.

Restriction Option 1: Operation Licensing

• The chip requires a license to operate. The license needs to be acquired from the external regulatory entity.
• The license could automatically expire after certain periods of time.

Restriction Option 2: Usage Limitations

• The chips could automatically limit
  • limit their usage in large cluster by coordinating and limiting external bandwidth
  • limit sensitive data access (this is the one I'm least skeptical of)
  • limit chips to only running approved code or models

Verification Option 1: Location Verification

• Can use speed of light limit to prove that a chip is within a certain distance by ping time
• Likely requires hundreds of trusted landmarks globally
• Queries take the form of cryptographic challenges issued against the chip's private key, to ensure that the responder is indeed the chip in question

Verification Option 2: Usage Verification

• Security dilemma
  • If you don't know what your rival's capabilities are, you need to race
  • This can be alleviated by reducing uncertainty re: capabilities
  • For AI, this could be known amount of compute used to train something
  • Can be verified through "hashed" data

Verification vs Monitoring

• "Monitoring" is bad; it lets you covertly see what's up. The target is being spied upon.
• "Verification" is ambiguous. The user can remotely attest to a third party verifier what they're doing.

The idea here is, we'd like to sell chips to people and not let them do "bad things", but still let them do "good things".

At this point, we have a ~15 minute, high-level intro to the fundamentals of cryptographic hash functions and public key cryptography.

Secure Boot

This is in service of discussing secure boot. This prevents authorized firmware, operating system or other software from running on a device to ensure that the chip will run only manufacturer-approved software. As a Linux user, I have strong, negative opinions of this strategy. I'm kind of curious to hear Stallman's take on this too, although I can probably imagine some of it.

Remote Attestation

Security Modules

• Many chips today have dedicated security modules, including a dedicated processor, that are responsible for handling private keys and security-related functions
• To implement an operating license, a security module would need to have the ability to limit or disable a chip's operations if the chip does not

Trusted Execution Environments

• The difference between security modules and TEEs is that TEEs create a protected environment on the main processor cores whereas a security module is a separate lower-performance processor specialized for security related tasks
• TEE = isolated but general environment, Security Module = dedicated, special-purpose module

Challenge 1: Privacy, Surveillance and Cybersecurity

• Immediate concerns for on-chip governance mechanisms is their potential to be misused
  • Unlawful surveillance
  • third party hackers taking advantage of insecure "back doors"

Challenge 2: Threat Models

• Minimally adversarial
  • Mostly cooperative situation, your users are aligned with you
• Covertly adversarial
  • Kind of cooperative; your users mostly want to be aligned with you, but will defect if they can do so undetected
• Openly adversarial
  • Not cooperative; your users are openly hostile and will defect whether or not they will be detected

Recommendations

• Establish government coordination
• Create commercial incentives
• Accelerate security R&D
• Plan for a staged roll-out and fund extensive red-teaming
• Coordinate with allies
• Encourage AI chip firms to move early