Interface bound agents #3: “The accuracy of state estimation is physically bound by the sensor interface”
In the last couple of posts, I have been applying ideas from Control theory to try to firm up the concept of an (AI) agent having a “narrow” (e.g. one servo motor) or “wide” interface (e.g. full text and audio, a physical body). The direction I am aiming in is to then argue on firm ground regarding whether a narrow interface is a limiting factor that is hard to overcome with raw intelligence.
In yesterday’s post, I started with a 5-layer breakdown of this interface/raw-intelligence dichotomy (sensors, input channel, “processing”, output channel, actuators), and1 managed to show that “processing” dissolves into the input channel as “state estimation”, and into the output channel as “action selection”. This neatly separates “the interface” as the physical sensors and actuators, with the “raw intelligence” part only able to see up to the sensor readings y(t) and control outputs u(t). I then foreshadowed that:
a few more steps of argument will probably let us say concretely that the accuracy of state estimation is physically bound by the sensor interface.
This is where I’m picking up from today.
The accuracy of state estimation is physically bound by the sensor interface
Two results from information theory
First, channel capacity: a noisy channel has a maximum information rate C = B × log₂(1 + S/N), where B is bandwidth and S/N is signal-to-noise ratio. Second, the data processing inequality: if you have a data processing chain X → Y → Z (true state → sensor reading → estimate), then your estimate can never contain more information about the true state than the sensor reading does.
Together: the sensor reading contains at most C bits/second about the true state. The estimate contains at most C bits/second about the true state, set by the physics of the sensor.
Concrete example
Here is a concrete example that doesn’t exactly load on the signal-to-noise ratio directly, to point out that this is a general “you can’t get something from nothing” idea.
Suppose an AI receives audio sampled at some frequency f Hz. By the Nyquist theorem, it can reconstruct signals up to f/2 Hz. So:
At f = 1 Hz: captures signals up to 0.5 Hz. Slow morse code is possible. Speech (~300-3400 Hz) is not possible.
At f = 100 Hz: signals up to 50 Hz. Fast morse code, still no speech.
At f = 8000 Hz: signals up to 4000 Hz. Telephone quality, speech is intelligible.
Similar logic applies for distinguishing speech from far away, or against background noise, etc. And: you could prevent an AI from understanding an audio signal by speeding it up and sending it in a short burst2.
The ability to drive the environment to a target state is physically bound by the actuator interface
This is again the dual of the sensor -> state estimation side.
Information theory again
On input: C bits/second bounds information in. On output: C bits/second bounds information out. I.e. if the control output u(t) is a 0 or 1, sent once every second, then you can can only inject C bits/second of directed change into the environment.
Concrete example
If a pick-and-place robot is trying to arrange 10 objects into specific positions on an 8x8 grid, it requires 20 3-bit numbers to specify the final configuration, so would take at least 60 seconds at 1 bit/second. If there is some vibration that makes objects drift off their grid spot within around 10 seconds, then it is impossible for the robot to drive the environment to the target state no matter how intelligent it is.
Conclusion
We have now demonstrated the obvious result we expected to demonstrate at the start of this series of posts, which arguably we’ve just been circling around the whole time. But, hopefully we’ve learned something in the process, and I am planning to continue over the next few days to ladder this back up to modern-AI-relevant systems. FOR INSTANCE:
Revisiting the idea of “recursive privilege escalation”. I.e. it is likely that an AI would be able to expand its interface over time? Is it possible to construct scenarios where this is definitionally impossible?
Applying the idea of “you can’t extract more info than the sensors pick up”, and “the environment decoheres faster than you can drive it back to your target state” to more harm-relevant AI scenarios, e.g. superpersuasion, autonomous weapons, etc.
E.g. the gist of an argument against superpersuasion: You get a few hundred bits/s in, and a few hundred bits/s out. The goal is to estimate the internal state of the brain well enough to reliably drive a person to take specific actions, which may take minutes/hours/days, during which time they will “decohere” due to dynamics of the environment (such as people telling them to stop). To me this makes “hypnotic” superpersuasion seem unlikely.
Trying to properly expand this analysis to non-linear systems (which I have been fast and loose with up to this point).
In a way that may or may not hold up
Potentially apocryphal anecdote: This was a trick used accidentally during the second world war. Signals sent via an enigma machine were usually typed out by hand and transmitted live (and these were intercepted and transcribed live by allied radio operators). For U-boats, this was a problem because they had to surface for the entire time the signal was sent (and in fact hold an antenna out of the open hatch), which exposed them to being targeted. To work around this, messages were recorded, and then sped up and transmitted as a short burst. As a side effect, this meant allied transcribers couldn’t write out the message (or at first even realise it was a message), until they cottoned on and also started recording them and replaying them slowed down.