The mirror that speaks: What teachers should know about AI

image thanks to Ronald Diel on Unsplash 

The dismissal, "AI is nothing but a next-word predictor" is becoming increasingly hard to reconcile with reality the more we use AI.

Yes, the architecture of the TRANSFORMER (the "T" in chat GTP) is applying a model of language to predict the next word. However, language is the conduit for ideas, the pipe that connects minds and creates all the incredible manifestations of culture. So any sufficiently good model of language – one that can pass the Turning test – must also have some dexterity in the conceptual space.

It appears this is an emergent property of a sufficiently complex system. Emergence describes properties that appear at a system's macro level but are not present in, or straightforwardly predictable from, its microscopic components. Think about a termite (I had millions of them eat part of my house, so this is a little traumatic). Each termite has about 900 k* neurons versus your 86 billion and yet they can craft a mound which has incredible properties -it breathes like a lung, yet is sealed with no permanent openings, it harnesses a fussy fungus for air conditioning and so much more... and yet no one termite holds the plans for any of this (or even partial plans) - it is an emergent phenomenon.

How are ideas captured in a bunch of numbers and simple mathematics? Let me take you through some of what we know (believe?) about AI so that as a teacher, you can better understand what's going on.

https://youtu.be/nQX2Obsgg2Y

While humans built AI, understanding how it works is an active area of research engaging many (and huge budgets) to probe how it is doing what it is doing. To me this is a signature of emergence. Don't mistake my energy for certainty - In the video I describe what researchers think is happening.

If you enjoyed this, you might like my other AI in Education posts.

REFERENCES

Firstly I owe a huge dept to Grant Sanderson's 3blue1brown youtube series on Large Language Models If you are finding my video like the wading pool and want to head for deeper water, I can recommend his 1 hour long presentation. It is designed for software engineers, but I found it is quite intelligible - after I re-watched some segments a few times!

Sanderson, G. [@GrantSanderson]. (2024). Visualizing transformers and attention | Talk for TNG Big Tech Day '24 [Video]. YouTube. https://www.youtube.com/watch?v=KJtZARuO3JY

Anderson, P.W. (1972). "More Is Different." Science 177(4047), 393–396. — The founding statement of scale-dependent laws.

Bedau, M.A. (1997). "Weak Emergence." Philosophical Perspectives 11, 375–399. — The definition most usable in science; emergence as incompressible simulation.

Camazine, S., Deneubourg, J.-L., Franks, N.R., Sneyd, J., Theraulaz, G., Bonabeau, E. (2001). Self-Organization in Biological Systems. Princeton UP. — The termite/ant/stigmergy literature, rigorously modelled.

Deacon, T.W. (1997). The Symbolic Species: The Co-evolution of Language and the Brain. Norton. — Symbolic reference as an emergent representational level.

Hutchins, E. (1995). Cognition in the Wild. MIT Press. — Cognition as a property of systems larger than single brains.

King, H., Ocko, S., & Mahadevan, L. (2015). Termite mounds harness diurnal temperature oscillations for ventilation. Proceedings of the National Academy of Sciences, 112(37), 11589–11593. https://doi.org/10.1073/pnas.1423242112

Martin, S. J., Funch, R. R., Hanson, P. R., & Yoo, E.-H. (2018). A vast 4,000-year-old spatial pattern of termite mounds. Current Biology, 28(22), R1292–R1293. https://doi.org/10.1016/j.cub.2018.09.061

Menzel, R., & Giurfa, M. (2001). Cognitive architecture of a mini-brain: The honeybee. Trends in Cognitive Sciences, 5(2), 62–71. https://doi.org/10.1016/S1364-6613(00)01601-6

*I used this reference to interpolate haw many neurons a termit as as the literature is sparse on this- my estimate is probably too high so please don;t quote it as fact

Turner, J. S., & Soar, R. C. (2008). Beyond biomimicry: What termites can tell us about realizing the living building. In I. Wallis, L. Bilan, M. Smith, & A. S. Kazi (Eds.), Proceedings of the First International Conference on Industrialised, Intelligent Construction (pp. 221–237). Loughborough University.