Main
Chaos: A Very Short Introduction
Chaos: A Very Short Introduction
Leonard A. Smith
5.0 /
4.0
How much do you like this book?
What’s the quality of the file?
Download the book for quality assessment
What’s the quality of the downloaded files?
Chaos exists in systems all around us. Even the simplest system can be subject to chaos, denying us accurate predictions of its behavior, and sometimes giving rise to astonishing structures of largescale order. Here, Leonard Smith shows that we all have an intuitive understanding of chaotic systems. He uses accessible math and physics to explain Chaos Theory, and points to numerous examples in philosophy and literature that illuminate the problems. This book provides a complete understanding of chaotic dynamics, using examples from mathematics, physics, philosophy, and the real world, with an explanation of why chaos is important and how it differs from the idea of randomness. The author's real life applications include the weather forecast, a pendulum, a coin toss, mass transit, politics, and the role of chaos in gambling and the stock market. Chaos represents a prime opportunity for mathematical lay people to finally get a clear understanding of this fascinating concept.
Categories:
Year:
2007
Publisher:
Oxford University Press, USA
Language:
english
Pages:
200
ISBN 10:
1429470062
ISBN 13:
9781429470063
Series:
Very Short Introductions
File:
PDF, 4.96 MB
Your tags:
Download (pdf, 4.96 MB)
 Open in Browser
 Checking other formats...
 Convert to EPUB
 Convert to FB2
 Convert to MOBI
 Convert to TXT
 Convert to RTF
 Converted file can differ from the original. If possible, download the file in its original format.
Report a problem
This book has a different problem? Report it to us
Check Yes if
Check Yes if
Check Yes if
Check Yes if
you were able to open the file
the file contains a book (comics are also acceptable)
the content of the book is acceptable
Title, Author and Language of the file match the book description. Ignore other fields as they are secondary!
Check No if
Check No if
Check No if
Check No if
 the file is damaged
 the file is DRM protected
 the file is not a book (e.g. executable, xls, html, xml)
 the file is an article
 the file is a book excerpt
 the file is a magazine
 the file is a test blank
 the file is a spam
you believe the content of the book is unacceptable and should be blocked
Title, Author or Language of the file do not match the book description. Ignore other fields.
Are you sure you want to report this book? Please specify the reason below
Change your answer
Thanks for your participation!
Together we will make our library even better
Together we will make our library even better
The file will be sent to your email address. It may take up to 15 minutes before you receive it.
The file will be sent to your Kindle account. It may takes up to 15 minutes before you received it.
Please note: you need to verify every book you want to send to your Kindle. Check your mailbox for the verification email from Amazon Kindle.
Please note: you need to verify every book you want to send to your Kindle. Check your mailbox for the verification email from Amazon Kindle.
Conversion to is in progress
Conversion to is failed
You may be interested in Powered by Rec2Me
Most frequent terms
chaos^{368}
models^{247}
map^{191}
systems^{190}
uncertainty^{181}
mathematical^{161}
forecast^{145}
observations^{123}
weather^{120}
chaotic^{115}
growth^{91}
forecasts^{89}
values^{84}
dynamics^{82}
ensemble^{80}
zero^{78}
initial^{66}
random^{62}
lyapunov^{62}
dynamical^{61}
linear^{60}
nonlinear^{59}
attractor^{59}
exponential^{59}
probability^{53}
average^{53}
logistic map^{52}
storm^{51}
deterministic^{49}
forecasting^{47}
dimensional^{45}
climate^{45}
statistics^{41}
rabbits^{41}
maps^{40}
predictability^{39}
parameter^{38}
distribution^{37}
exponential growth^{37}
attractors^{36}
computers^{36}
fractals^{35}
mathematical models^{35}
fractal^{35}
loop^{35}
bits^{35}
burns^{34}
dimension^{34}
mathematicians^{32}
iteration^{32}
behaviour^{32}
lyapunov exponent^{31}
odds^{31}
stochastic^{31}
dynamical systems^{30}
periodic^{29}
mathematics^{29}
doubling^{28}
lyapunov exponents^{28}
galton^{27}
Related Booklists
0 comments
You can write a book review and share your experiences. Other readers will always be interested in your opinion of the books you've read. Whether you've loved the book or not, if you give your honest and detailed thoughts then people will find new books that are right for them.
1

2

Chaos: A Very Short Introduction Very Short Introductions are for anyone wanting a stimulating and accessible way in to a new subject. They are written by experts, and have been published in more than 25 languages worldwide. The series began in 1995, and now represents a wide variety of topics in history, philosophy, religion, science, and the humanities. Over the next few years it will grow to a library of around 200 volumes – a Very Short Introduction to everything from ancient Egypt and Indian philosophy to conceptual art and cosmology. Very Short Introductions available now: ANARCHISM Colin Ward ANCIENT EGYPT Ian Shaw ANCIENT PHILOSOPHY Julia Annas ANCIENT WARFARE Harry Sidebottom ANGLICANISM Mark Chapman THE ANGLOSAXON AGE John Blair ANIMAL RIGHTS David DeGrazia ARCHAEOLOGY Paul Bahn ARCHITECTURE Andrew Ballantyne ARISTOTLE Jonathan Barnes ART HISTORY Dana Arnold ART THEORY Cynthia Freeland THE HISTORY OF ASTRONOMY Michael Hoskin Atheism Julian Baggini Augustine Henry Chadwick BARTHES Jonathan Culler THE BIBLE John Riches THE BRAIN Michael O’Shea BRITISH POLITICS Anthony Wright Buddha Michael Carrithers BUDDHISM Damien Keown BUDDHIST ETHICS Damien Keown CAPITALISM James Fulcher THE CELTS Barry Cunliffe CHAOS Leonard Smith CHOICE THEORY Michael Allingham CHRISTIAN ART Beth Williamson CHRISTIANITY Linda Woodhead CLASSICS Mary Beard and John Henderson CLAUSEWITZ Michael Howard THE COLD WAR Robert McMahon CONSCIOUSNESS Susan Blackmore CONTEMPORARY ART Julian Stallabrass Continental Philosophy Simon Critchley COSMOLOGY Peter Coles THE CRUSADES Christopher Tyerman CRYPTOGRAPHY Fred Piper and Sean Murphy DADA AND SURREALISM David Hopkins Darwin Jonathan Howard THE DEAD SEA SCROLLS Timothy Lim Democracy Bernard Crick DESCARTES Tom Sorell DESIGN John Heskett DINOSAURS David Norman DREAMING J. Allan Hobson DRUGS Leslie Iversen THE EARTH Martin Redfern ECONOMICS Partha Dasgupta EGYPTIAN MYTH Geraldine Pinch EIGHTEENTHCENTURY BRITAIN Paul Langford THE ELEMENTS Philip Ball EMOTION Dylan Evans EMPIRE Stephen; Howe ENGELS Terrell Carver Ethics Simon Blackburn The European Union John Pinder EVOLUTION Brian and Deborah Charlesworth EXISTENTIALISM Thomas Flynn FASCISM Kevin Passmore FEMINISM Margaret Walters THE FIRST WORLD WAR Michael Howard FOSSILS Keith Thomson FOUCAULT Gary Gutting THE FRENCH REVOLUTION William Doyle FREE WILL Thomas Pink Freud Anthony Storr FUNDAMENTALISM Malise Ruthven Galileo Stillman Drake Gandhi Bhikhu Parekh GLOBAL CATASTROPHES Bill McGuire GLOBALIZATION Manfred Steger GLOBAL WARMING Mark Maslin HABERMAS James Gordon Finlayson HEGEL Peter Singer HEIDEGGER Michael Inwood HIEROGLYPHS Penelope Wilson HINDUISM Kim Knott HISTORY John H. Arnold HOBBES Richard Tuck HUMAN EVOLUTION Bernard Wood HUME A. J. Ayer IDEOLOGY Michael Freeden Indian Philosophy Sue Hamilton Intelligence Ian J. Deary INTERNATIONAL MIGRATION Khalid Koser ISLAM Malise Ruthven JOURNALISM Ian Hargreaves JUDAISM Norman Solomon Jung Anthony Stevens KAFKA Ritchie Robertson KANT Roger Scruton KIERKEGAARD Patrick Gardiner THE KORAN Michael Cook LINGUISTICS Peter Matthews LITERARY THEORY Jonathan Culler LOCKE John Dunn LOGIC Graham Priest MACHIAVELLI Quentin Skinner THE MARQUIS DE SADE John Phillips MARX Peter Singer MATHEMATICS Timothy Gowers MEDICAL ETHICS Tony Hope MEDIEVAL BRITAIN John Gillingham and Ralph A. Grifﬁths MODERN ART David Cottington MODERN IRELAND Senia Pašeta MOLECULES Philip Ball MUSIC Nicholas Cook Myth Robert A. Segal NATIONALISM Steven Grosby NEWTON Robert Iliffe NIETZSCHE Michael Tanner NINETEENTHCENTURY BRITAIN Christopher Harvie and H. C. G. Matthew NORTHERN IRELAND Marc Mulholland PARTICLE PHYSICS Frank Close paul E. P. Sanders Philosophy Edward Craig PHILOSOPHY OF LAW Raymond Wacks PHILOSOPHY OF SCIENCE Samir Okasha PHOTOGRAPHY Steve Edwards PLATO Julia Annas POLITICS Kenneth Minogue POLITICAL PHILOSOPHY David Miller POSTCOLONIALISM Robert Young POSTMODERNISM Christopher Butler POSTSTRUCTURALISM Catherine Belsey PREHISTORY Chris Gosden PRESOCRATIC PHILOSOPHY Catherine Osborne Psychology Gillian Butler and Freda McManus PSYCHIATRY Tom Burns QUANTUM THEORY John Polkinghorne THE RENAISSANCE Jerry Brotton RENAISSANCE ART Geraldine A. Johnson ROMAN BRITAIN Peter Salway THE ROMAN EMPIRE Christopher Kelly ROUSSEAU Robert Wokler RUSSELL A. C. Grayling RUSSIAN LITERATURE Catriona Kelly THE RUSSIAN REVOLUTION S. A. Smith SCHIZOPHRENIA Chris Frith and Eve Johnstone SCHOPENHAUER Christopher Janaway SHAKESPEARE Germaine Greer SIKHISM Eleanor Nesbitt SOCIAL AND CULTURAL ANTHROPOLOGY John Monaghan and Peter Just SOCIALISM Michael Newman SOCIOLOGY Steve Bruce Socrates C. C. W. Taylor THE SPANISH CIVIL WAR Helen Graham SPINOZA Roger Scruton STUART BRITAIN John Morrill TERRORISM Charles Townshend THEOLOGY David F. Ford THE HISTORY OF TIME Leofranc HolfordStrevens TRAGEDY Adrian Poole THE TUDORS John Guy TWENTIETHCENTURY BRITAIN Kenneth O. Morgan THE VIKINGS Julian D. Richards Wittgenstein A. C. Grayling WORLD MUSIC Philip Bohlman THE WORLD TRADE ORGANIZATION Amrita Narlikar Available soon: AFRICAN HISTORY John Parker and Richard Rathbone CHILD DEVELOPMENT Richard Grifﬁn CITIZENSHIP Richard Bellamy HIV/AIDS Alan Whiteside HUMAN RIGHTS Andrew Clapham INTERNATIONAL RELATIONS Paul Wilkinson RACISM Ali Rattansi For more information visit our web site www.oup.co.uk/general/vsi/ Leonard A. Smith CHAOS A Very Short Introduction 1 3 Great Clarendon Street, Oxford o x 2 6 d p Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide in Oxford New York Auckland Cape Town Dar es Salaam Hong Kong Karachi Kuala Lumpur Madrid Melbourne Mexico City Nairobi New Delhi Shanghai Taipei Toronto With ofﬁces in Argentina Austria Brazil Chile Czech Republic France Greece Guatemala Hungary Italy Japan Poland Portugal Singapore South Korea Switzerland Thailand Turkey Ukraine Vietnam Oxford is a registered trade mark of Oxford University Press in the UK and in certain other countries Published in the United States by Oxford University Press Inc., New York © Leonard A. Smith 2007 The moral rights of the author have been asserted Database right Oxford University Press (maker) First published as a Very Short Introduction 2007 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior permission in writing of Oxford University Press, or as expressly permitted by law, or under terms agreed with the appropriate reprographics rights organizations. Enquiries concerning reproduction outside the scope of the above should be sent to the Rights Department, Oxford University Press, at the address above You must not circulate this book in any other binding or cover and you must impose this same condition on any acquirer British Library Cataloguing in Publication Data Data available Library of Congress Cataloging in Publication Data Data available Typeset by ReﬁneCatch Ltd, Bungay, Suffolk Printed in Great Britain by Ashford Colour Press Ltd, Gosport, Hampshire 978–0–19–285378–3 1 3 5 7 9 10 8 6 4 2 To the memory of Dave Paul Debeer, A real physicist, a true friend. This page intentionally left blank Contents Acknowledgements xi Preface xii List of illustrations xv 1 2 3 The emergence of chaos 1 Exponential growth, nonlinearity, common sense 22 Chaos in context: determinism, randomness, and noise 4 5 6 7 8 9 10 11 33 Chaos in mathematical models 58 Fractals, strange attractors, and dimension(s) 76 Quantifying the dynamics of uncertainty 87 Real numbers, real observations, and computers 104 Sorry, wrong number: statistics and chaos 112 Predictability: does chaos constrain our forecasts? Applied chaos: can we see through our models? Philosophy in chaos 154 Glossary 163 123 132 Further reading 169 Index 173 Acknowledgements This book would not have been possible without my parents, of course, but I owe a greater debt than most to their faith, doubt, and hope, and to the love and patience of a, b, and c. Professionally my greatest debt is to Ed Spiegel, a father of chaos and my thesis Professor, mentor, and friend. I also proﬁted immensely from having the chance to discuss some of these ideas with Jim Berger, Robert Bishop, David Broomhead, Neil Gordon, Julian Hunt, Kevin Judd, Joe Keller, Ed Lorenz, Bob May, Michael Mackey, Tim Palmer, Itamar Procaccia, Colin Sparrow, James Theiler, John Wheeler, and Christine Ziehmann. I am happy to acknowledge discussions with, and the support of, the Master and Fellows of Pembroke College, Oxford. Lastly and largely, I’d like to acknowledge my debt to my students, they know who they are. I am never sure how to react upon overhearing an exchange like: ‘Did you know she was Lenny’s student?’, ‘Oh, that explains a lot.’ Sorry guys: blame Spiegel. Preface The ‘chaos’ introduced in the following pages reﬂects phenomena in mathematics and the sciences, systems where (without cheating) small differences in the way things are now have huge consequences in the way things will be in the future. It would be cheating, of course, if things just happened randomly, or if everything continually exploded forever. This book traces out the remarkable richness that follows from three simple constraints, which we’ll call sensitivity, determinism, and recurrence. These constraints allow mathematical chaos: behaviour that looks random, but is not random. When allowed a bit of uncertainty, presumed to be the active ingredient of forecasting, chaos has reignited a centuriesold debate on the nature of the world. The book is selfcontained, deﬁning these terms as they are encountered. My aim is to show the what, where, and how of chaos; sidestepping any topics of ‘why’ which require an advanced mathematical background. Luckily, the description of chaos and forecasting lends itself to a visual, geometric understanding; our examination of chaos will take us to the coalface of predictability without equations, revealing open questions of active scientiﬁc research into the weather, climate, and other realworld phenomena of interest. Recent popular interest in the science of chaos has evolved differently than did the explosion of interest in science a century ago when special relativity hit a popular nerve that was to throb for decades. Why was the public reaction to science’s embrace of mathematical chaos different? Perhaps one distinction is that most of us already knew that, sometimes, very small differences can have huge effects. The concept now called ‘chaos’ has its origins both in science ﬁction and in science fact. Indeed, these ideas were well grounded in ﬁction before they were accepted as fact: perhaps the public were already well versed in the implications of chaos, while the scientists remained in denial? Great scientists and mathematicians had sufﬁcient courage and insight to foresee the coming of chaos, but until recently mainstream science required a good solution to be well behaved: fractal objects and chaotic curves were considered not only deviant, but the sign of badly posed questions. For a mathematician, few charges carry more shame than the suggestion that one’s professional life has been spent on a badly posed question. Some scientists still dislike problems whose results are expected to be irreproducible even in theory. The solutions that chaos requires have only become widely acceptable in scientiﬁc circles recently, and the public enjoyed the ‘I told you so’ glee usually claimed by the ‘experts’. This also suggests why chaos, while widely nurtured in mathematics and the sciences, took root within applied sciences like meteorology and astronomy. The applied sciences are driven by a desire to understand and predict reality, a desire that overcame the niceties of whatever the formal mathematics of the day. This required rare individuals who could span the divide between our models of the world and the world as it is without convoluting the two; who could distinguish the mathematics from the reality and thereby extend the mathematics. As in all Very Short Introductions, restrictions on space require entire research programmes to be glossed over or omitted; I present a few recurring themes in context, rather than a series of shallow descriptions. My apologies to those whose work I have omitted, and my thanks to Luciana O’Flaherty (my editor), Wendy Parker, and Lyn Grove for help in distinguishing between what was most interesting to me and what I might make interesting to the reader. How to read this introduction While there is some mathematics in this book, there are no equations more complicated than X = 2. Jargon is less easy to discard. Words in bold italics you will have to come to grips with; these are terms that are central to chaos, brief deﬁnitions of these words can be found in the Glossary at the end of the book. Italics is used both for emphasis and to signal jargon needed for the next page or so, but which is unlikely to recur often throughout the book. Any questions that haunt you would be welcome online at http:// cats.lse.ac.uk/forum/ on the discussion forum VSI Chaos. More information on these terms can be found rapidly at Wikipedia http://www.wikipedia.org/ and http://cats.lse.ac.uk/preditcabilitywiki/ , and in the Further reading. List of illustrations 1 The ﬁrst weather map ever published in a newspaper, prepared by Galton in 1875 6 A graph comparing Fibonacci numbers and exponential growth 26 7 © The Times/NI Syndication Limited 2 Galton’s original sketch of the Galton Board 9 3 The Times headline following the Burns’ Day storm in 1990 13 © The Times/NI Syndication Limited 1990/John Frost Newspapers 4 Modern weather map showing the Burns’ Day storm and a twodayahead forecast 14 5 The Cheat with the Ace of Diamonds, c.1645, by Georges de la Tour 19 Louvre, Paris. © Photo12.com/ Oronoz 7 A chaotic time series from the Full Logistic Map 39 8 Six mathematical maps 40 9 Points collapsing onto four attractors of the Logistic Map 48 10 The evolution of uncertainty under the Yule Map 52 11 Period doubling behaviour in the Logistic Map 12 61 A variety of more complicated behaviours in the Logistic Map 62 13 Threedimensional bifurcation diagram and the collapse toward attractors in the Logistic Map 63 21 14 The Lorenz attractor and the MooreSpiegel attractor 67 15 The evolution of uncertainty in the Lorenz System 22 Predictable chaos as seen in four iterations of the same mouse ensemble under the Baker’s Map and a Baker’s Apprentice Map 100 16 68 The Hénon attractor and a twodimensional slice of the MooreSpiegel attractor 70 17 A variety of behaviours from the HénonHeilies System 72 18 The Fournier Universe, as illustrated by Fournier 78 19 Time series from the stochastic Middle Thirds IFS Map and the deterministic Tripling Tent Map 82 20 A close look at the Hénon attractor, showing fractal structure Schematic diagrams showing the action of the Baker’s Map and a Baker’s Apprentice Map 98 23 Card trick revealing the limitations of digital computers 108 24 Two views of data from Machete’s electric circuit, suggestive of Takens’ Theorem 118 25 The Not A Galton Board 26 An illustration of using analogues to make a forecast 134 27 The state space of a climate model 136 Crown Copyright 28 Richardson’s dream © F. Schuiten 84 128 137 29 Twodayahead ECMWF ensemble forecasts of the Burns’ Day storm 140 30 Four ensemble forecasts of the Machete’s MooreSpiegel Circuit 150 Figures 7, 8, 9, 11, 12, 13, 19, and 20 were produced with the assistance of Hailiang Du. Figures 24 and 30 were produced with the assistance of Reason Machete. Figures 4 and 29 were produced with the assistance of Martin Leutbecher with data kindly made available by the European Centre for MediumRange Weather Forecasting. Figure 27 is after M. Hume et al., The UKIP02 Scientiﬁc Report, Tyndal Centre, University of East Anglia, Norwich, UK. The publisher and the author apologize for any errors or omissions in the above list. If contacted they will be pleased to rectify these at the earliest opportunity. This page intentionally left blank Chapter 1 The emergence of chaos Embedded in the mud, glistening green and gold and black, was a butterﬂy, very beautiful and very dead. It fell to the ﬂoor, an exquisite thing, a small thing that could upset balances and knock down a line of small dominoes and then big dominoes and then gigantic dominoes, all down the years across Time. Ray Bradbury (1952) Three hallmarks of mathematical chaos The ‘butterﬂy effect’ has become a popular slogan of chaos. But is it really so surprising that minor details sometimes have major impacts? Sometimes the proverbial minor detail is taken to be the difference between a world with some butterﬂy and an alternative universe that is exactly like the ﬁrst, except that the butterﬂy is absent; as a result of this small difference, the worlds soon come to differ dramatically from one another. The mathematical version of this concept is known as sensitive dependence. Chaotic systems not only exhibit sensitive dependence, but two other properties as well: they are deterministic, and they are nonlinear. In this chapter, we’ll see what these words mean and how these concepts came into science. Chaos is important, in part, because it helps us to cope with 1 unstable systems by improving our ability to describe, to understand, perhaps even to forecast them. Indeed, one of the myths of chaos we will debunk is that chaos makes forecasting a useless task. In an alternative but equally popular butterﬂy story, there is one world where a butterﬂy ﬂaps its wings and another world where it does not. This small difference means a tornado appears in only one of these two worlds, linking chaos to uncertainty and prediction: in which world are we? Chaos is the name given to the mechanism which allows such rapid growth of uncertainty in our mathematical models. The image of chaos amplifying uncertainty and confounding forecasts will be a recurring theme throughout this Introduction. Chaos Whispers of chaos Warnings of chaos are everywhere, even in the nursery. The warning that a kingdom could be lost for the want of a nail can be traced back to the 14th century; the following version of the familiar nursery rhyme was published in Poor Richard’s Almanack in 1758 by Benjamin Franklin: For want of a nail the shoe was lost, For want of a shoe the horse was lost, and for want of a horse the rider was lost, being overtaken and slain by the enemy, all for the want of a horseshoe nail. We do not seek to explain the seed of instability with chaos, but rather to describe the growth of uncertainty after the initial seed is sown. In this case, explaining how it came to be that the rider was lost due to a missing nail, not the fact that the nail had gone missing. In fact, of course, there either was a nail or there was not. But Poor Richard tells us that if the nail hadn’t been lost, then the kingdom wouldn’t have been lost either. We will often explore the properties of chaotic systems by considering the impact of slightly different situations. 2 The study of chaos is common in applied sciences like astronomy, meteorology, population biology, and economics. Sciences making accurate observations of the world along with quantitative predictions have provided the main players in the development of chaos since the time of Isaac Newton. According to Newton’s Laws, the future of the solar system is completely determined by its current state. The 19thcentury scientist Pierre Laplace elevated this determinism to a key place in science. A world is deterministic if its current state completely deﬁnes its future. In 1820, Laplace conjured up an entity now known as ‘Laplace’s demon’; in doing so, he linked determinism and the ability to predict in principle to the very notion of success in science. We may regard the present state of the universe as the effect of its past and the cause of its future. An intellect which at a certain positions of all items of which nature is composed, if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes. Note that Laplace had the foresight to give his demon three properties: exact knowledge of the Laws of Nature (‘all the forces’), the ability to take a snapshot of the exact state of the universe (‘all the positions’), and inﬁnite computational resources (‘an intellect vast enough to submit these data to analysis’). For Laplace’s demon, chaos poses no barrier to prediction. Throughout this Introduction, we will consider the impact of removing one or more of these gifts. From the time of Newton until the close of the 19th century, most scientists were also meteorologists. Chaos and meteorology are closely linked by the meteorologists’ interest in the role uncertainty plays in weather forecasts. Benjamin Franklin’s interest in 3 The emergence of chaos moment would know all forces that set nature in motion, and all Chaos meteorology extended far beyond his famous experiment of ﬂying a kite in a thunderstorm. He is credited with noting the general movement of the weather from west towards the east and testing this theory by writing letters from Philadelphia to cities further east. Although the letters took longer to arrive than the weather, these are arguably early weather forecasts. Laplace himself discovered the law describing the decrease of atmospheric pressure with height. He also made fundamental contributions to the theory of errors: when we make an observation, the measurement is never exact in a mathematical sense, so there is always some uncertainty as to the ‘True’ value. Scientists often say that any uncertainty in an observation is due to noise, without really deﬁning exactly what the noise is, other than that which obscures our vision of whatever we are trying to measure, be it the length of a table, the number of rabbits in a garden, or the midday temperature. Noise gives rise to observational uncertainty, chaos helps us to understand how small uncertainties can become large uncertainties, once we have a model for the noise. Some of the insights gleaned from chaos lie in clarifying the role(s) noise plays in the dynamics of uncertainty in the quantitative sciences. Noise has become much more interesting, as the study of chaos forces us to look again at what we might mean by the concept of a ‘True’ value. Twenty years after Laplace’s book on probability theory appeared, Edgar Allan Poe provided an early reference to what we would now call chaos in the atmosphere. He noted that merely moving our hands would affect the atmosphere all the way around the planet. Poe then went on to echo Laplace, stating that the mathematicians of the Earth could compute the progress of this handwaving ‘impulse’, as it spread out and forever altered the state of the atmosphere. Of course, it is up to us whether or not we choose to wave our hands: free will offers another source of seeds that chaos might nurture. In 1831, between the publication of Laplace’s science and Poe’s 4 all manner of insects, vultures, inﬁnite billions of life forms are thrown into chaos and destruction . . . Step on a mouse and you leave your print, like a Grand Canyon, across Eternity. Queen Elizabeth might never be born, Washington might not cross the Delaware, there might never be a United States at all. So be careful. Stay on the Path. Never step off! Needless to say, someone does step off the Path, crushing to death a beautiful little green and black butterﬂy. We can only consider these ‘what if’ experiments within the ﬁctions of mathematics or literature, since we have access to only one realization of reality. The origins of the term ‘butterﬂy effect’ are appropriately shrouded 5 The emergence of chaos ﬁction, Captain Robert Fitzroy took the young Charles Darwin on his voyage of discovery. The observations made on this voyage led Darwin to his theory of natural selection. Evolution and chaos have more in common than one might think. First, when it comes to language, both ‘evolution’ and ‘chaos’ are used simultaneously to refer both to phenomena to be explained and to the theories that are supposed to do the explaining. This often leads to confusion between the description and the object described (as in ‘confusing the map with the territory’). Throughout this Introduction we will see that confusing our mathematical models with the reality they aim to describe muddles the discussion of both. Second, looking more deeply, it may be that some ecosystems evolve as if they were chaotic systems, as it may well be the case that small differences in the environment have immense impacts. And evolution has contributed to the discussion of chaos as well. This chapter’s opening quote comes from Ray Bradbury’s ‘A Sound Like Thunder’, in which timetravelling big game hunters accidentally kill a butterﬂy, and ﬁnd the future a different place when they return to it. The characters in the story imagine the impact of killing a mouse, its death cascading through generations of lost mice, foxes, and lions, and: in mystery. Bradbury’s 1952 story predates a series of scientiﬁc papers on chaos published in the early 1960s. The meteorologist Ed Lorenz once invoked sea gulls’ wings as the agent of change, although the title of that seminar was not his own. And one of his early computergenerated pictures of a chaotic system does resemble a butterﬂy. But whatever the incarnation of the ‘small difference’, whether it be a missing horse shoe nail, a butterﬂy, a sea gull, or most recently, a mosquito ‘squished’ by Homer Simpson, the idea that small differences can have huge effects is not new. Although silent regarding the origin of the small difference, chaos provides a description for its rapid ampliﬁcation to kingdomshattering proportions, and thus is closely tied to forecasting and predictability. Chaos The ﬁrst weather forecasts Like every ship’s captain of the time, Fitzroy had a deep interest in the weather. He developed a barometer which was easier to use onboard ship, and it is hard to overestimate the value of a barometer to a captain lacking access to satellite images and radio reports. Major storms are associated with low atmospheric pressure; by providing a quantitative measurement of the pressure, and thus how fast it is changing, a barometer can give lifesaving information on what is likely to be over the horizon. Later in life, Fitzroy became the ﬁrst head of what would become the UK Meteorological Ofﬁce and exploited the newly deployed telegraph to gather observations and issue summaries of the current state of the weather across Britain. The telegraph allowed weather information to outrun the weather itself for the ﬁrst time. Working with LeVerrier of France, who became famous for using Newton’s Laws to discover two new planets, Fitzroy contributed to the ﬁrst international efforts at realtime weather forecasting. These forecasts were severely criticized by Darwin’s cousin, statistician Francis Galton, who himself published the ﬁrst weather chart in the London Times in 1875, reproduced in Figure 1. 6 1. The ﬁrst weather chart ever published in a newspaper. Prepared by Francis Galton, it appeared in the London Times on 31 March 1875 Chaos If uncertainty due to errors of observation provides the seed that chaos nurtures, then understanding such uncertainty can help us better cope with chaos. Like Laplace, Galton was interested in the ‘theory of errors’ in the widest sense. To illustrate the ubiquitous ‘bellshaped curve’ which so often seems to reﬂect measurement errors, Galton created the ‘quincunx’, which is now called a Galton Board; the most common version is shown on the left side of Figure 2. By pouring lead shot into the quincunx, Galton simulated a random system in which each piece of shot has a 50:50 chance of going to either side of every ‘nail’ that it meets, giving rise to a bellshaped distribution of lead. Note there is more here than the oneoff ﬂap of a butterﬂy wing: the paths of two nearby pieces of lead may stay together or diverge at each level. We shall return to Galton Boards in Chapter 9, but we will use random numbers from the bellshaped curve as a model for noise many times before then. The bellshape can be seen at the bottom of the Galton Board on the left of Figure 2, and we will ﬁnd a smoother version towards the top of Figure 10. The study of chaos yields new insight into why weather forecasts remain unreliable after almost two centuries. Is it due to our missing minor details in today’s weather which then have major impacts on tomorrow’s weather? Or is it because our methods, while better than Fitzroy’s, remain imperfect? Poe’s early atmospheric incarnation of the butterﬂy effect is complete with the idea that science could, if perfect, predict everything physical. Yet the fact that sensitive dependence would make detailed forecasts of the weather difﬁcult, and perhaps even limit the scope of physics, has been recognized within both science and ﬁction for some time. In 1874, the physicist James Clerk Maxwell noted that a sense of proportion tended to accompany success in a science: This is only true when small variations in the initial circumstances produce only small variations in the ﬁnal state of the system. In a great many physical phenomena this condition is satisﬁed; but there are other cases in which a small initial variation may produce a very 8 The emergence of chaos 2. Galton’s 1889 schematic drawings of what are now called ‘Galton Boards’ great change in the ﬁnal state of the system, as when the displacement of the ‘points’ causes a railway train to run into another instead of keeping its proper course. This example is again atypical of chaos in that it is ‘oneoff’ sensitivity, but it does serve to distinguish sensitivity and uncertainty: this sensitivity is no threat as long as there is no uncertainty in the position of the points, or in which train is on which track. Consider pouring a glass of water near a ridge in the 9 Chaos Rocky Mountains. On one side of this continental divide the water ﬁnds its way into the Colorado River and to the Paciﬁc Ocean, on the other side the Mississippi River and eventually the Atlantic Ocean. Moving the glass one way or the other illustrates sensitivity: a small change in the position of the glass means a particular molecule of water ends up in a different ocean. Our uncertainty in the position of the glass might restrict our ability to predict which ocean that molecule of water will end up in, but only if that uncertainty crosses the line of the continental divide. Of course, if we were really trying to do this, we would have to question whether any such mathematical line actually divided continents, as well as the other adventures the molecule of water might have which could prevent it reaching the ocean. Usually, chaos involves much more than a single oneoff ‘tripping point’; it tends to more closely resemble a water molecule that repeatedly evaporates and falls in a region where there are continental divides all over the place. Nonlinearity is deﬁned by what it is not (it is not linear). This kind of deﬁnition invites confusion: how would one go about deﬁning a biology of nonelephants? The basic idea to hold in mind now is that a nonlinear system will show a disproportionate response: the impact of adding a second straw to a camel’s back could be much bigger (or much smaller) than the impact of the ﬁrst straw. Linear systems always respond proportionately. Nonlinear systems need not, giving nonlinearity a critical role in the origin of sensitive dependence. The Burns’ Day storm But Mousie, thou art no thy lane, In proving foresight may be vain: The bestlaid schemes o mice an men Gang aft agley, An lea’e us nought but grief an pain, For promis’d joy! 10 Still thou art blest, compar’d wi me! The present only toucheth thee: But och! I backward cast my e’e, On prospects drear! An forward, tho I canna see, I guess an fear! Robert Burns, ‘To A Mouse’ (1785) In contrast, the Great Storm of 1987 is famous for a BBC television meteorologist’s broadcast the night before, telling people not to worry about rumours from France that a hurricane was about to strike England. Both storms, in fact, managed gusts of over 100 miles per hour, and the Burns’ Day storm caused much greater loss of life; yet 20 years after the event, the Great Storm of 1987 is much more often discussed, perhaps exactly because the Burns’ Day storm was well forecast. The story leading up to this forecast beautifully illustrates a different way that chaos in our 11 The emergence of chaos Burns’ poem praises the mouse for its ability to live only in the present, not knowing the pain of unfulﬁlled expectations nor the dread of uncertainty in what is yet to pass. And Burns was writing in the 18th century, when mice and men laid their plans with little assistance from computing machines. While foresight may be pain, meteorologists struggle to foresee tomorrow’s likely weather every day. Sometimes it works. In 1990, on the anniversary of Burns’ birth, a major storm ripped through northern Europe, including the British Isles, causing signiﬁcant property damage and loss of life. The centre of the storm passed over Burns’ home town in Scotland, and it became known as the Burns’ Day storm. A weather chart reﬂecting the storm at noon on 25 January is shown in the top panel of Figure 4 (page 14). Ninetyseven people died in northern Europe, about half of this number in Britain, making it the highest death toll of any storm in 40 years; about 3 million trees were blown down, and total insurance costs reached £2 billion. Yet the Burns’ Day storm has not joined the rogues’ gallery of famously failed forecasts: it was well forecast by the Met Ofﬁce. Chaos models can impact our lives without invoking alternate worlds, some with and some without butterﬂies. In the early morning of 24 January 1990, two ships in the midAtlantic sent routine meteorological observations from positions that happened to straddle the centre of what would become the Burns’ Day storm. The forecast models run with these observations give a ﬁne forecast of the storm. Running the model again after the event showed that when these observations are omitted, the model predicts a weaker storm in the wrong place. Because the Burns’ Day storm struck during the day, the failure to provide forewarning would have had a huge impact on loss of life, so here we have an example where a few observations, had they not been made, would have changed the forecast and hence the course of human events. Of course, an ocean weather ship is harder to misplace than a horse shoe nail. There is more to this story, and to see its relevance we need to look into how weather models ‘work’. Operational weather forecasting is a remarkable phenomenon in and of itself. Every day, observations are taken in the most remote locations possible, and then communicated and shared among national meteorological ofﬁces around the globe. Many different nations use this data to run their computer models. Sometimes an observation is subject to plain old mistakes, like putting the temperature in the box for wind speed, or a typo, or a glitch in transition. To keep these mistakes from corrupting the forecast, incoming observations are subject to quality control: observations that disagree with what the model is expecting (given its last forecast) can be rejected, especially if there are no independent, nearby observations to lend support to them. It is a welllaid plan. Of course, there are rarely any ‘nearby’ observations of any sort in the middle of the Atlantic, and the ship observations showed the development of a storm that the model had not predicted would be there, so the computer’s automatic quality control program simply rejected these observations. 12 3. Headline from The Times the day after the Burns’ Day storm 4. A modern weather chart reﬂecting the Burns’ Day storm as seen through a weather model (top) and a twodayahead forecast targeting the same time showing a fairly pleasant day (bottom) Luckily, the computer was overruled. An intervention forecaster was on duty and realized that these observations were of great value. His job was to intervene when the computer did something obviously silly, as computers are prone to do. In this case, he tricked the computer into accepting the observations. Whether or not to take this action is a judgement call: there was no way to know at the time which action would yield a better forecast. The computer was ‘tricked’, the observation was used. The storm was forecast, and lives were saved. 15 The emergence of chaos There are two takehome messages here: the ﬁrst is that when our models are chaotic then small changes in our observations can have large impacts on the quality of our foresight. An accountant looking to reduce costs and computing the typical beneﬁt of one particular observation from any particular weather station is likely to vastly underestimate the value of a future report from one of those weather stations that falls at the right place at the right time, and similarly the value of the intervention forecaster, who often has to do nothing, literally. The second is that the Burns’ Day forecast illustrates something a bit different from the butterﬂy effect. Mathematical models allow us to worry about what the real future will bring not by considering possible worlds, of which there may be only one, but by contrasting different simulations of our model, of which there can be as many as we can afford. As Burns might appreciate, science gives us new ways to guess and new things to fear. The butterﬂy effect contrasts different worlds: one world with the nail and another world without that nail. The Burns effect places the focus ﬁrmly on us and our attempts to make rational decisions in the real world given only collections of different simulations under various imperfect models. The failure to distinguish between reality and our models, between observations and mathematics, arguably between an empirical fact and scientiﬁc ﬁction, is the root of much confusion regarding chaos both by the public and among scientists. It was research into nonlinearity and chaos that clariﬁed yet again how import this distinction remains. In Chapter 10, we will return to take a deeper look at how today’s weather forecasters would have used insights from their understanding of chaos when making a forecast for this event. We have now touched on the three properties found in chaotic mathematical systems: chaotic systems are nonlinear, they are deterministic, and they are unstable in that they display sensitivity to initial condition. In the chapters that follow we will constrain them further, but our real interests lie not only in the mathematics of chaos, but also in what it can tell us about the real world. Chaos and the real world: predictability and a 21stcentury demon There is no more greater an error in science, than to believe that just because some mathematical calculation has been completed, some aspect of Nature is certain. Chaos Alfred North Whitehead (1953) What implications does chaos hold for our everyday lives? Chaos impacts the ways and means of weather forecasting, which affect us directly through the weather, and indirectly through economic consequences both of the weather and of the forecasts themselves. Chaos also plays a role in questions of climate change and our ability to foresee the strength and impacts of global warming. While there are many other things that we forecast, weather and climate can be used to represent shortrange forecasting and longrange modelling, respectively. ‘When is the next solar eclipse?’ would be a weatherlike question in astronomy, while ‘Is the solar system stable?’ would be a climatelike question. In ﬁnance, when to buy 100 shares of a given stock is a weatherlike question, while a climatelike question might address whether to invest in the stock market or real estate. Chaos has also had a major impact on the sciences, forcing a close reexamination of what scientists mean by the words ‘error’ and ‘uncertainty’ and how these meanings change when applied to our 16 world and our models. As Whitehead noted, it is dangerous to interpret our mathematical models as if they somehow governed the real world. Arguably, the most interesting impacts of chaos are not really new, but the mathematical developments of the last 50 years have cast many old questions into a new light. For instance, what impact would uncertainty have on a 21stcentury incarnation of Laplace’s demon which could not escape observational noise? In his 1927 Gifford Lectures, Sir Arthur Eddington went to the heart of the problem of chaos: some things are trivial to predict, especially if they have to do with mathematics itself, while other things seem predictable, sometimes: A total eclipse of the sun, visible in Cornwall is prophesied for 11 August 1999 . . . I might venture to predict that 2 + 2 will be equal to 4 even in 1999 . . . The prediction of the weather this time next year . . . is not likely to ever become practicable . . . We should require extremely detailed knowledge of present conditions, since a small local deviation can exert an everexpanding inﬂuence. We must examine the state of the sun . . . be forewarned of volcanic eruptions, . . . , a coal strike . . . , a lighted match idly thrown away . . . 17 The emergence of chaos Consider an intelligence that knew all the laws of nature precisely and had good, but imperfect, observations of an isolated chaotic system over an arbitrarily long time. Such an agent – even if sufﬁciently vast to subject all this data to computationally exact analysis – could not determine the current state of the system and thus the present, as well as the future, would remain uncertain in her eyes. While our agent could not predict the future exactly, the future would hold no real surprises for her, as she could see what could and what could not happen, and would know the probability of any future event: the predictability of the world she could see. Uncertainty of the present will translate into wellquantiﬁed uncertainty in the future, if her model is perfect. Our best models of the solar system are chaotic, and our best models of the weather appear to be chaotic: yet why was Eddington conﬁdent in 1928 that the 1999 solar eclipse would occur? And equally conﬁdent that no weather forecast a year in advance would ever be accurate? In Chapter 10 we will see how modern weather forecasting techniques designed to better cope with chaos helped me to see that solar eclipse. Chaos When paradigms collide: chaos and controversy One of the things that has made working in chaos interesting over the last 20 years has been the friction generated when different ways of looking at the world converge on the same set of observations. Chaos has given rise to a certain amount of controversy. The studies that gave birth to chaos have revolutionized not only the way professional weather forecasters forecast but even what a forecast consists of. These new ideas often run counter to traditional statistical modelling methods, and still produce both heat and light on how best to model the real world. This battle is broken into skirmishes by the nature of the ﬁeld and our level of understanding in the particular system of which a question is asked, be it the population of voles in Scandinavia, a mathematical calculation to quantify chaos, the number of spots on the Sun’s surface, the price of oil delivered next month, tomorrow’s maximum temperature, or the date of the last ever solar eclipse. The skirmishes are interesting, but chaos offers deeper insights even when both sides are ﬁghting for traditional advantage, say, the ‘best’ model. Here studies of chaos have redeﬁned the high ground: today we are forced to reconsider new deﬁnitions for what constitutes the best model, or even a ‘good’ model. Arguably, we must give up the idea of approaching Truth, or at least deﬁne a wholly new way of measuring our distance from it. The study of chaos motivates us to establish utility without any hope of achieving perfection, and to give up many obvious home truths of forecasting, 18 like the naı̈ve idea that a good forecast consists of a prediction that is close to the target. This did not appear naı̈ve before we understood the implications of chaos. La Tour’s realistic vision of science in the real world 5. The Cheat with the Ace of Diamonds, by Georges de la Tour, painted about 1645 19 The emergence of chaos To close this chapter, we illustrate how chaos can force us to reconsider what constitutes a good model, and revise our beliefs as to what is ultimately responsible for our forecast failures. This impact is felt by scientists and mathematicians alike, but the reconsideration will vary depending on the individual’s point of view and the empirical system under study. The situation is nicely personiﬁed in Figure 5, a French baroque painting by Georges de la Tour showing a card game from the 17th century. La Tour was arguably a realist with a sense of humour. He was fond of fortune telling and games of chance, especially those in which chance played a somewhat lesser role than the participants happened to believe. In theory, chaos can play exactly this role. We will interpret Chaos this painting to show a mathematician, a physicist, a statistician, and a philosopher engaged in an exercise of skill, dexterity, insight, and computational prowess; this is arguably a description for doing science, but the task at hand here is a game of poker. Exactly who is who in the painting will remain open, as we will return to these personiﬁcations of natural science throughout the book. The insights chaos yields vary with the perspective of the viewer, but a few observations are in order. The impeccably groomed young man on the right is engaged in careful calculations, no doubt a probability forecast of some nature; he is currently in possession of a handsome collection of gold coins on the table. The dealer plays a critical role, without her there is no game to be played; she provides the very language within which we communicate, yet she seems to be in nonverbal communication with the handmaiden. The role of the handmaiden is less clear; she is perhaps tangential, but then again the provision of wine will inﬂuence the game, and she herself may feature as a distraction. The roguish character in ramshackle dress with bows untied is clearly concerned with the real world, not mere appearances in some model of it; his left hand is extracting one of several aces of diamonds from his belt, which he is about to introduce into the game. What then do the ‘probabilities’ calculated by the young man count for, if, in fact, he is not playing the game his mathematical model describes? And how deep is the insight of our rogue? His glance is directed to us, suggesting that he knows we can see his actions, perhaps even that he realizes that he is in a painting? The story of chaos is important because it enables us to see the world from the perspective of each of these players. Are we merely developing the mathematical language with which the game is played? Are we risking economic ruin by overinterpreting some potentially useful model while losing sight of the fact that it, like all models, is imperfect? Are we only observing the big picture, not entering the game directly but sometimes providing an interesting distraction? Or are we manipulating those things we can change, 20 acknowledging the risks of model inadequacy, and perhaps even our own limitations, due to being within the system? To answer these questions we must ﬁrst examine several of the many jargons of science in order to be able to see how chaos emerged from the noise of traditional linear statistics to vie for roles both in understanding and in predicting complicated realworld systems. Before the nonlinear dynamics of chaos were widely recognized within science, these questions fell primarily in the domain of the philosophers; today they reach out via our mathematical models to physical scientists and working forecasters, changing the statistics of decision support and even impacting politicians and policy makers. The emergence of chaos 21 Chapter 2 Exponential growth, nonlinearity, common sense One of the most pervasive myths about chaotic systems is that they are impossible to predict. To expose the fallacy of this myth, we must understand how uncertainty in a forecast grows as we predict further and further into the future. In this chapter we investigate the origin and meaning of exponential growth, since on average a small uncertainty will grow exponentially fast in a chaotic system. There is a sense in which this phenomenon really does imply a ‘faster’ growth of uncertainty than that found in our traditional ideas of how error and uncertainty grow as we forecast further into the future. Nevertheless, chaos can be easy to predict, sometimes. Chess, rice, and Leonardo’s rabbits: exponential growth An ofttold story about the origin of the game of chess illustrates nicely the speed of exponential growth. The story goes that a king of ancient Persia was so pleased when ﬁrst presented with the game that he wanted to reward the game’s creator, Sissa Ben Dahir. A chess board has 64 squares arranged in an 8 by 8 pattern; for his reward, Ben Dahir requested what seemed a quite modest sum of rice determined using the new chess board: one grain of rice was to be put on the ﬁrst square of the board, two to be put on the second, four for the third, eight for the fourth, and so on, doubling the number on each square until the 64th was reached. A 22 mathematician will often call any rule for generating one number from another one a mathematical map, so we’ll refer to this simple rule (‘double the current value to generate the next value’) as the Rice Map. Before working out just how much rice Ben Dahir has asked for, let us consider the case of linear growth where we have one grain on the ﬁrst square, two on the second square, three on the third, and so on until we need 64 for the last square. In this case we have a total of 64 + 63 + 62 + . . . + 3 + 2 + 1, or around 1,000 grains. Just for comparison, a 1 kilogram bag of rice contains a few tens of thousands of grains. By comparing the amount of rice on a given square in the case of linear growth with the amount of rice on the same square in the case of exponential growth, we quickly see that exponential is much faster than linear growth: on the fourth square we already have twice as many grains in the exponential case as in the linear case (8 in the ﬁrst, only 4 in the second), and by the eighth square, at the end of the ﬁrst row, the exponential case has 16 times more! Soon thereafter we have the astronomical numbers. 23 Exponential growth, nonlinearity, common sense The Rice Map requires one grain for the ﬁrst square, then two for the second, four for the third, then 8, 16, 32, 64, and 128 for the last square of the ﬁrst row. On the third square of the second row, we pass 1,000 and before the end of the second row there is a square which exhausts our bag of rice. To ﬁll the next square alone will require another entire bag, the following square two bags, and so on. Some square in the third row will require a volume of rice comparable to a small house, and we will have enough rice to ﬁll the Royal Albert Hall well before the end of the ﬁfth row. Finally, the 64th square alone will require billions and billions, or to be exact, 263 (= 9, 223, 372, 036, 854, 775, 808) grains, for a total of 18,446,744,073,709,551,615 grains. That is a nontrivial quantity of rice! It is something like the entire world’s rice production over two millennia. Exponential growth quickly grows out of all proportion. Chaos Of course, we hid the values of some parameters in the example above: we could have made the linear growth faster by adding not one additional grain for each square, but instead, say, 1,000 additional grains. This parameter, the number of additional grains, deﬁnes the constant of proportionality between the number of the square and the number of grains on that square, and gives us the slope of the linear relationship between them. There is also a parameter in the exponential case: on each step we increased the number of grains by a factor of two, but it could have been a factor of three, or a factor of one and a half. One of the surprising things about exponential growth is that whatever the values of these parameters, there will come a time at which exponential growth surpasses any linear growth, and will soon thereafter dwarf linear growth, no matter how fast the linear growth is. Our ultimate interest is not in rice on a chess board, but in the dynamics of uncertainty in time. Not just the growth of a population, but the growth of our uncertainty in a forecast of the future size of that population. In the forecasting context, there will come a time at which an exponentially growing uncertainty which is very small today will surpass a linearly growing uncertainty which is today much larger. And the same thing happens when contrasting exponential growth with growth proportional to the square of time, or to the cube of time, or to time raised to any power (in symbols: steady exponential growth will eventually surpass the growth proportional to t2 or t3 or tn for any value of n.). It is for this reason among others that exponential growth is mathematically distinguished, and taken to provide a benchmark for deﬁning chaos. It has also contributed to the widespread but fundamentally mistaken impression that chaotic systems are hopelessly unpredictable. Ben Dahir’s chess board illustrates that there is a deep sense in which exponential growth is faster than linear growth. To place this in the context of forecasting, we move forward a few hundred years in time and a few hundred miles northwest, from Persia to Italy. 24 So what does this ‘population dynamic’ look like? In the ﬁrst month we have one immature pair, in the second month we have one mature pair, in the third month we have one mature pair and a new immature pair, in the fourth month we have two mature pairs and one immature pair, in the ﬁfth month we have three mature pairs and two immature. If we count up all the pairs each month, the numbers are 1, 1, 2, 3, 5, 8, 13, 21 . . . . Leonardo noted that the next number in the series is always the sum of the previous two numbers (1 + 1 = 2, 2 + 1 = 3, 3 + 2 = 5, . . . ) which makes sense, as the previous number is the number we had last month (in our model all rabbits survive no matter how many there are), and the penultimate number is the number of mature pairs (and thus the number of new pairs arriving this month). Now it gets a bit tedious to write ‘and in the sixth month we have 12 pairs of rabbits’, so scientists often use a shorthand X for the number of pairs of rabbits and X6 to denote the number of pairs in month six. And since the series 1, 1, 2, 3, 5, 8, . . . reﬂects how the population of rabbits evolves in time, this series and others like it are called time series. The Rabbit Map is deﬁned by the rule: 25 Exponential growth, nonlinearity, common sense At the beginning of the 13th century, Leonardo of Pisa posed a question of population dynamics: given a newborn pair of rabbits in a large, lush, walled garden, how many pairs of rabbits will we have in one year if their nature is for each mature pair to breed and produce a new pair every month, and newborn rabbits mature in their second month? In the ﬁrst month we have one juvenile pair. In the second month this pair matures and breeds to produce a new pair in the third month. So in the third month, we have one mature pair and one newborn pair. In the fourth month we once again have one new born pair from the original pair of rabbits and now two mature pairs for a total of three pairs. In the ﬁfth month, two new pairs are born (one from each mature pair), and we have three mature pairs for a total of ﬁve pairs. And so on. Add the previous value of X to the current value of X, and take the sum as the new value of X. Chaos The numbers in the series 1, 1, 2, 3, 5, 8, 13, 21, 34 . . . are called Fibonacci numbers (Fibonacci was a nickname of Leonardo of Pisa), and they arise again and again in nature: in the structure of sunﬂowers, pine cones, and pineapples. They are of interest here because they illustrate exponential growth in time, almost. The crosses in Figure 6 are Fibonacci’s points – the rabbit population as a function of time – while the solid line reﬂects two raised to the power λt, or in symbols 2λt , where t is the time in months and λ is our ﬁrst exponent. Exponents which multiply time in the superscript are a useful way of quantifying uniform exponential growth. In this case, λ is equal to the logarithm of a number called the golden mean, a very special number which is discussed in the Very Short Introduction to Mathematics. 6. The series of crosses showing the number of pairs of rabbits each month (Fibonacci numbers); the smooth curve they lie near is the related exponential growth 26 So how can Leonardo’s rabbits help us to get a feel for the growth of forecast uncertainty? Like all observations, counting the number of rabbits in a garden is subject to error; as we saw in Chapter 1, observational uncertainties are said to be caused by noise. Imagine that Leonardo failed to notice a pair of mature rabbits also in the garden in the ﬁrst month; in that case, the number of pairs actually in the garden would have been 2, 3, 5, 8, 13, . . . The error in the original forecast (1, 1, 2, 3, 5, 8 . . . ) would be the difference between the Truth and that forecast, namely: 1, 2, 3, 5 . . . (again, the Fibonacci series). In month 12, this error has reached a very noticeable 146 pairs of rabbits! A small error in the initial number of rabbits results in a very large error in the forecast. In fact, the error is growing exponentially in time. This has many implications. Consider the impact of the exponential error growth on the uncertainty of our forecasts. Let us again contrast linear growth and exponential growth. Let’s assume that, for a price, we can reduce the uncertainty in the initial observation that we use in generating 27 Exponential growth, nonlinearity, common sense The ﬁrst thing to notice about Figure 6 is that the points lie close to the curve. The exponential curve is special in mathematics because it reﬂects a function whose increase is proportional to its current value. The larger it gets, the faster it grows. It makes sense that something like this function would describe the dynamics of Leonardo’s rabbit population since the number of rabbits next month is more or less proportional to the number of rabbits this month. The second thing to notice about the ﬁgure is that the points do not lie on the curve. The curve is a good model for Fibonacci’s Rabbit Map, but it is not perfect: at the end of each month the number of rabbits is always a whole number and, while the curve may be close to the correct whole number, it is not exactly equal to it. As the months go by and the population grows, the curve gets closer and closer to each Fibonacci number, but it never reaches them. This concept of getting closer and closer but never quite arriving is one that will come up again and again in this book. Chaos our forecast. If the error growth is linear, and we reduce our initial uncertainty by a factor of ten, then we can forecast the system ten times longer before our uncertainty exceeds the same threshold. If we reduce the initial uncertainty by a factor of 1,000, then we can get forecasts of the same quality 1,000 times longer. This is an advantage of linear models. Or, more accurately, this is an apparent advantage of studying only linear systems. By contrast, if the model is nonlinear and the uncertainty grows exponentially, then we may reduce our initial uncertainty by a factor of ten yet only be able to forecast twice as long with the same accuracy. In that case, assuming the exponential growth in uncertainty is uniform in time, reducing the uncertainty by a factor of 1,000 will only increase our forecast range at the same accuracy by a factor of eight. Now reducing the uncertainty in a measurement is rarely free (we have to hire someone else to count the rabbits a second time), and large reductions of uncertainty can be expensive, so when uncertainty grows exponentially fast, the cost skyrockets. Attempting to achieve our forecast goals by reducing uncertainty in initial conditions can be tremendously expensive. Luckily, there is an alternative that allows us to accept the simple fact that we can never be certain that any observation has not been corrupted by noise. In the case of rabbits or grains of rice, it seems there really is a fact of the matter, a whole number that reﬂects the correct answer. If we reduce the uncertainty in this initial condition to zero then we can predict without error. But can we ever really be certain of the initial condition? Might there not be another bunny hiding in the noise? While our best guess is that there is one pair in the garden, there might be two, or three, or more (or perhaps zero). When we are uncertain of the initial condition, we can examine the diversity of forecasts under our model by making an ensemble of forecasts: one forecast started from each initial condition we think plausible. So one member of the ensemble will start with X equal to one, another ensemble member will start with X equals two, and so on. How should we divide our limited resources between computing 28 more ensemble members and making better observations of the current number of rabbits in the garden? But what if we were measuring something that is not a whole number, like temperature, or the position of a planet? And is temperature in an imperfect weather model exactly the same thing as temperature in the real world? It was these questions that initially interested our philosopher in chaos. First, we should consider the more pressing question of why rabbits have not taken over the world in the 9,000 months since 1202? Stretching, folding, and the growth of uncertainty The study of chaos lends credence to the meteorological maxim that no forecast is complete without a useful estimate of forecast uncertainty: if we know our initial condition is uncertain then we are not only interested in the prediction per se, but equally in learning what the likely forecast error will be. Forecast error for any 29 Exponential growth, nonlinearity, common sense In the Rabbit Map, differences between the forecasts of different members of the ensemble will grow exponentially fast, but with an ensemble forecast we can see just how different they are and use this as a measure of our uncertainty in the number of rabbits we expect at any given time. In addition, if we carefully count the number of rabbits after a few months, we can all but rule out some of the individual ensemble members. Each of these ensemble members was started from some estimate of the number of rabbits that were in the garden originally, so ruling an ensemble member out in effect gives us more information about the original number of rabbits. Of course, this information need only prove accurate if our model is literally perfect, meaning, in this case, that our Rabbit Map captures the reproductive behaviour and longevity of our rabbits exactly. But if our model is perfect, then we can use future observations to learn about the past; this process is called noise reduction. If it turns out that our model is not perfect, then we may end up with incoherent results. Exponential growth: an example from Miss Nagel’s third grade class A few months ago, I received an email written by an old friend of mine from elementary school. It contained another email that had originated from a third grader in North Carolina whose class was studying geography. It requested that everyone who read the email send a reply to the school stating where they lived, and the class would locate that place on a school globe. It also requested that each reader pass on the email to ten friends. I did not forward the message to anyone, but I did write an email to Miss Nagel’s class stating that I was in Oxford, Chaos England. I also suggested that they tell their mathematics teacher about their experiment and use it as an example to illustrate exponential growth: if they sent the message to ten people, and the next day each of them sent it to ten more people, that would be 100 on day three, 1,000 on day four, and more emails than there are email addresses within a week or so. In a real system, exponential growth cannot go on forever: eventually we run out of rice, or garden space, or new email addresses. It is often the resources that limit growth: even a lush garden provides only a ﬁnite amount of rabbit food. There are limits to growth which bound populations, if not our models of populations. I never found out whether Miss Nagel’s class learned their lesson in exponential growth. The only answer I ever received was an automated reply stating that the school’s email inbox had exceeded its quota and had been closed. 30 real system should not grow without limit; even if we start with a small error like one grain or one rabbit, the forecast error will not grow arbitrarily large (unless we have a very naı̈ve forecaster), but will saturate near some limiting value, as would the population itself. Our mathematician has a way to avoid ludicrously large forecast errors (other than naı̈veté), namely by making the initial uncertainty inﬁnitesimally small – smaller than any number you can think of, yet greater than zero. Such an uncertainty will stay inﬁnitesimally small for all time, even if it grows exponentially fast. Whenever our model goes into nevernever land (suggesting values where no data have ever gone before), then something is likely to give, unless something in our model has already broken. Often, as our uncertainty grows too large, it starts to fold back on itself. Imagine kneading dough, or a toffee machine continuously stretching and folding toffee. An imaginary line of toffee connecting two very nearby grains of sugar will grow longer and longer as these two grains separate under the action of the machine, but before it becomes bigger than the machine itself, this line will be folded back into itself, forming a horrible tangle. The distance between the grains of sugar will stop growing, even as the string of toffee connecting them continues to grow longer and longer, becoming a more and more complicated tangle. The toffee machine gives us a way to envision limits to the growth of prediction error whenever our model is perfect. In this case, the error is the growing distance 31 Exponential growth, nonlinearity, common sense Physical factors, like the total amount of rabbit food in the garden or the amount of disk space on an email system, limit growth in practice. The limits are intuitive even if we do not know exactly what causes them: I think I have lost my keys in the car park; of course they might be several miles from there, but it is exceedingly unlikely that they are farther away than the moon. I do not need to understand or believe the laws of gravity to appreciate this. Similarly, weather forecasters are rarely more than 100 degrees off, even for a forecast one year in advance! Even inadequate models can usually be constrained so that their forecast errors are bounded. Chaos between the True state and our best guess of that state: any exponential growth of error would correspond only to the rapid initial growth of the string of toffee. But if our forecasts are not going to zoom away towards inﬁnity (the toffee must stay in the machine, only a ﬁnite number of rabbits will ﬁt in the garden, and the like), then eventually the line connecting Truth and our forecast will be folded over on itself. There is simply nowhere else for it to grow into. In many ways, identifying the movement of a grain of sugar in the toffee machine with the evolution of the state of a chaotic system in three dimensions is a useful way to visualize chaotic motion. We want to require a sense of containment for chaos, since it is hardly surprising that it is difﬁcult to predict things that are ﬂying apart to inﬁnity, but we do not want to impose so strict a condition as requiring a forecast to never exceed some limited value, no matter how big that value might be. As a compromise, we require the system to come back to the vicinity of its current state at some point in the future, and to do so again and again. It can take as long as it wants to come back, and we can deﬁne coming back to mean returning closer to the current point than we have ever seen it return before. If this happens, then the trajectory is said to be recurrent. The toffee again provides an analogy: if the motion was chaotic and we wait long enough, our two grains of sugar will again come back close together, and each will pass close to where it was at the beginning of the experiment, assuming no one turns off the machine in the meantime. 32 Chapter 3 Chaos in context: determinism, randomness, and noise All linear systems resemble one another, each nonlinear system is nonlinear in its own way. After Tolstoy’s Anna Karenina Dynamical systems Chaos is a property of dynamical systems. And a dynamical system is nothing more than a source of changing observations: Fibonacci’s imaginary garden with its rabbits, the Earth’s atmosphere as reﬂected by a thermometer at London’s Heathrow airport, the economy as observed through the price of IBM stock, a computer program simulating the orbit of the moon and printing out the date and location of each future solar eclipse. There are at least three different kinds of dynamical systems. Chaos is most easily deﬁned in mathematical dynamical systems. These systems consist of a rule: you put a number in and you get a new number out, which you put back in, to get yet a newer number out, which you put back in. And so on. This process is called iteration. The number of rabbits each month in Fibonacci’s imaginary garden is a perfect example of a time series from this kind of system. A second type of dynamical system is found in the empirical world of the physicist, the biologist, or the stock market trader. Here, our sequence of observations consists of noisy measurements of reality, 33 Chaos which are fundamentally different from the noisefree numbers of the Rabbit Map. In these physical dynamical systems – the Earth’s atmosphere and Scandinavia’s vole population, for example – numbers represent the state, whereas in the Rabbit Map they were the state. To avoid needless confusion, it is useful to distinguish a third case when a digital computer performs the arithmetic speciﬁed by a mathematical dynamical system; we will call this a computer simulation – computer programs that produce TV weather forecasts are a common example. It is important to remember that these are different kinds of systems and that each is a different beast: our best equations for the weather differ from our best computer models based on those equations, and both of these systems differ from the real thing the Earth’s atmosphere itself. Confusingly, the numbers from each of our three types of systems are called time series, and we must constantly struggle to keep in mind the distinction between what these are time series of: a number of imaginary rabbits, the True temperature at the airport (if such a thing exists), a measurement representing that temperature, and a computer simulation of that temperature. The extent to which these differences are important depends on what we aim to do. Like la Tour’s card players, scientists, mathematicians, statisticians, and philosophers each have different talents and aims. The physicist may aim to describe the observations with a mathematical model, perhaps testing the model by using it to predict future observations. Our physicist is willing to sacriﬁce mathematical tractability for physical relevance. Mathematicians like to prove things that are true for a wide range of systems, but they value proof so highly that they often do not care how widely they must restrict that range to have it; one should almost always be wary whenever a mathematician is heard to say ‘almost every’. Our physicist must be careful not to forget this and confuse mathematical utility with physical relevance; physical intuitions should not be biased by the properties of ‘wellunderstood’ systems designed only for their mathematical tractability. 34 Mathematical dynamical systems and attractors We commonly ﬁnd four different types of behaviour in time series. They can (i) grind to a halt and more or less repeat the same ﬁxed number over and over again, (ii) bounce around in a closed loop like a broken record, periodically repeating the same pattern: exactly the same series of numbers over and over, (iii) move in a loop that has more than one period and so does not quite repeat exactly but comes close, like the moment of high tide drifting through the time of day, or (iv) forever jump about wildly, or perhaps even calmly, displaying no obvious pattern. The fourth type looks random, yet looks can be deceiving. Chaos can look random but it is not random. In fact, as we have learned to see better, chaos often does not even look all that random to us anymore. In the next few pages we will introduce several more maps, though perhaps without the rice or rabbits. We need these maps in order to generate interesting artefacts for our tour in search of the various types of behaviour just noted. Some of these maps were generated by mathematicians for this very purpose, although our physicist might argue, with reason, that a given map was derived by simplifying physical laws. In truth, the maps are simple enough to have each come about in several different ways. 35 Chaos in context: determinism, randomness, and noise Our statistician is interested in describing interesting statistics from the time series of real observations and in studying the properties of dynamical systems that generate time series which look like the observations, always taking care to make as few assumptions as possible. Finally, our philosopher questions the relationships among the underlying physical system that we claim generated the observations, the observations themselves, and the mathematical models or statistical techniques that we created to analyse them. For example, she is interested in what we can know about the relationship between the temperature we measure and the true temperature (if such a thing exists), and in whether the limits on our knowledge are merely practical difﬁculties we might resolve or limits in principle that we can never overcome. Chaos Before we can produce a time series by iterating a map, we need some number to start with. This ﬁrst number is called an initial condition, an initial state that we deﬁne, discover, or arrange for our system to be. As in Chapter 2, we adopt the symbol X as shorthand for a state of our system. The collection of all possible states X is called the state space. For Fibonacci’s imaginary rabbits, this would be the set of all whole numbers. Suppose our time series is from a model of the average number of insects per square mile at midsummer each year. In that case, X is just a number and the state space, being the collection of all possible states, is then a line. It sometimes takes more than one number to deﬁne the state, and if so X will have more than one component. In predatorprey models, for instance, the populations of both are required and X has two components: it is a vector. When X is a vector containing both the number of voles (prey) and the number of weasels (predators) on the ﬁrst of January each year, then the state space will be a twodimensional surface – a plane – that contains all pairs of numbers. If X has three components (say, voles, weasels, and annual snowfall), then the state space is a threedimensional space containing all triplets of numbers. Of course, there is no reason to stop at three components; although the pictures become more challenging to draw in higher dimensions, modern weather models have over 10,000,000 components. For a mathematical system, X can even be a continuous ﬁeld, like the height of the surface of the ocean or the temperature at every point on the surface of the Earth. However, our observations of physical systems will never be more complicated than a vector, and since we will only measure a ﬁnite number of things, our observations will always be ﬁnitedimensional vectors. For the time being, we will consider the case in which X is a simple number, such as onehalf. Recalling that a mathematical map is just a rule that transforms one set of values into the next set of values, you can deﬁne the Quadrupling Map by the rule: Multiply X by four to form the new value of X. 36 Given an initial condition, like X equals onehalf, this mathematical dynamical system produces a time series of values of X, in this case ½ × 4 = 2, 2 × 4 = 8, 8 × 4 = 32 . . . and the time series is 0.5, 2, 8, 32, 128, 512, 2048 . . . And so on. This series just gets bigger and bigger and, dynamically speaking, that is not so interesting. If a time series of X grows without limit like this one does, we call it unbounded. In order to get a dynamical system where X is bounded, we’ll take a second example, the Quartering Map: Take X divided by four as the new X In the Full Logistic Map, time series from almost every X bounces around irregularly between zero and one forever: 37 Chaos in context: determinism, randomness, and noise Starting at X = ½ yields the time series 1/8, 1/32, 1/128, . . . . At ﬁrst sight, this is not very exciting since X rapidly shrinks towards zero. But in fact, the Quartering Map has been carefully designed to illustrate special mathematical properties. The origin – the state X = 0 – is a ﬁxed point: if we start there we will never leave, since zero divided by four is again zero. The origin is also our ﬁrst attractor; under the Quartering Map the origin is the inevitable if unreachable destination: if we start with some other value of X, we never actually make it to the attractor, although we get close as the number of iterations increases without limit. How close? Arbitrarily close. As close as you like. Inﬁnitesimally close, meaning closer than any number you can name. Name a number, any number, and we can work out how many iterations are required after which X will remain closer to zero than that number. Getting arbitrarily close to an attractor as time goes on while never quite reaching it is a common feature of many time series from nonlinear systems. The pendulum provides a physical analogue: each swing will be smaller than the last, an effect we blame on air resistance and friction. The analogue of the attractor in this case is the motionless pendulum hanging straight down. We will have more to say about attractors after we have added a few more dynamical systems to our menagerie. Subtract X2 from X, multiply the difference by four and take the result as the new X. Chaos If we multiply components of state variables by other components, things become nonlinear. What is the time series in this case if we again start with X equals onehalf? Starting with ½, X minus X2 is ¼, times four is one, so our new value is one. Continuing with X now equal to one, we have X minus X2 is zero. But four times zero is always zero, so we’ll get zeros forever. And our time series is 0.5, 1, 0, 0, 0 . . . This does not blow up, but it is hardly exciting; recall the warning about ‘almost every’. The order of the numbers in a time series is important, whether the series reﬂects monthly values of Fibanocci’s rabbits or iterations of the Full Logistic Map. Using the shorthand suggested in Chapter 2, we will write X5 for the ﬁfth new value of X, and X0 for the initial state (or observation), and in general Xi for the ith value. Whether we are iterating the map or taking observations, i is always an integer and is often called ‘time’. In the Full Logistic Map with X0 is equal to 0.5, X1 is equal to 1, X2 is 0, X3 is 0, X4 is 0, and Xi will be zero for all i greater than four as well. So the origin is again a ﬁxed point. But under the Full Logistic Map small values of X grow (you can check this with a hand calculator), X = 0 is unstable and so the origin is not an attractor. A time series started near the origin is in fact unlikely to take one of the ﬁrst three options noted at the opening of this section, but to bounce about chaotically forever. Figure 7 shows a time series starting near X0 equals 0.876; it represents a chaotic time series from the Full Logistic Map. But look at it closely: does it really look completely unpredictable? It looks like small values of X are followed by small values of X, and that there is a tendency for the time series to linger whenever it is near threequarters. Our physicist would look at this series and expect it to be predictable at least sometimes, while, after a few 38 calculations, our statistician might even declare it random. Although we can see this structure, the most common statistical tests cannot. A menagerie of maps The rule that deﬁnes a map can be stated either in words, or as an equation, or in a graph. Each panel of Figure 8 deﬁnes the rule graphically. To use the graph, ﬁnd the current value of X on the horizontal axis, and then move directly upward until you hit the curve; the value of this point on the curve on the vertical axis is the new value of X. The Full Logistic Map is shown graphically in Figure 8 (b), while the Quarter Map is in panel (a). An easy way of using the graph to see if a ﬁxed point is unstable is to look at the slope of the map at the ﬁxed point: if the slope is steeper than 45 degrees (either up or down); then the ﬁxed point is 39 Chaos in context: determinism, randomness, and noise 7. A chaotic time series from the Full Logistic Map starting near X0 equals 0.876. Note the series is visibly predictable whenever X is near zero and threequarters 8. Graphical presentation of the (a) Quarter Map, (b) Full Logistic Map, (c) Shift Map, (d) Tent Map, (e) Tripling Tent Map, and (f) the MoranRicker Map Chaos unstable. In the Quartering Map the slope is less than one everywhere, while for the Full Logistic Map the slope near the origin is greater than one. Here small but nonzero values of X grow with each iteration but only as long as they stay sufﬁciently small (the slope near ½ is zero). As we will see below, for almost every initial condition between zero and one, the time series displays true mathematical chaos. The Full Logistic Map is pretty simple; chaos is pretty common. To see if a mathematical system is deterministic merely requires checking carefully whether carrying out the rule requires a random number. If not, then the dynamical system is deterministic: every time we put the same value of X in, we get the same new value of X out. If the rule requires (really requires) a random number, then the system is random, also called stochastic. With a stochastic system, even if we iterate exactly the same initial condition we expect the details of the next value of X and thus the time series to be different. Looking back at their deﬁnitions, we see that the three maps deﬁned above are each deterministic; their future time series is completely determined by the initial condition, hence the name ‘deterministic system’. Our philosopher would point out that just knowing X is not enough, we also need to know the mathematical system and we have to have the power to do exact calculations with it. These were the three gifts Laplace ensured his demon possessed 200 years ago. Our ﬁrst stochastic dynamical system is the AC Map: Divide X by four, then subtract ½ and add a random number R to get the new X. The AC Map is a stochastic system since applying the rule requires access to a supply of random numbers. In fact, the rule above is incomplete, since it does not specify how to get R. To complete the deﬁnition we must add something like: for R on each iteration, pick a number between zero and one in a manner that each number is equally likely to be chosen, which implies that R will be uniformly 42 distributed between zero and one and that the probability of the next value of R falling in an interval of values is proportional to the width of that interval. In the AC Map, each value of R is used within the map, but there is another class of random maps – called Iterated Function Systems, or IFS for short – which appear to use the value of R not in a formula but to make a decision as to what to do. One example is the Middle Thirds IFS Map, which will come in handy later when we try to work out the properties of maps from the time series that they generate. The Middle Thirds IFS Map is: Take a random number R from a uniform distribution between zero and one. If R is less than a half, take X/3 as the new X Otherwise take 1 – X/3 as the new X. So now we have a few mathematical systems, and we can easily tell if they are deterministic or stochastic. What about computer simulations? Digital computer simulations are always deterministic. And as we shall see in Chapter 7, the time series from a digital computer is either on an endless loop of values repeating itself periodically, over and over again, or it is on its way towards such a loop. This ﬁrst part of a time series in which no value is repeated, the trajectory is evolving towards a periodic loop but has not reached it, is called a transient. In mathematical circles, this word is something of an insult, since 43 Chaos in context: determinism, randomness, and noise What rule do we use to pick R? It could not be a deterministic rule, since then R would not be random. Arguably, there is no ﬁnite rule for generating values of R. This has nothing to do with needing uniform numbers between zero and one. We’d have the same problem if we wanted to generate random numbers which mimicked Galton’s ‘bellshape’ distribution. We will have to rely on our statistician to somehow get us the random numbers we need; hereafter we’ll just state whether they have a uniform distribution or the bellshaped distribution. Chaos mathematicians prefer to work with longlived things, not mere transients. While mathematicians avoid transients, physical scientists may never see anything else and, as it turns out, digital computers cannot maintain them. The digital computers that have proven critical in advancing our understanding of chaos cannot, ironically, display true mathematical chaos themselves. Neither can a digital computer generate random numbers. The socalled random number generators on digital computers and hand calculators are, in fact, only pseudorandom number generators; one of the earliest of these generators was even based on the Full Logistic Map! The difference between mathematical chaos and computer simulations, like that between random numbers and pseudorandom numbers, exempliﬁes the difference between our mathematical systems and our computer simulations. The maps in Figure 8 are not there by chance. Mathematicians often construct systems in such a way that it will be relatively simple for them to illustrate some mathematical point or allow the application of some speciﬁc manipulation – a word they sometimes use to obscure technical sleight of hand. The really complicated maps – including the ones used to guide spacecraft and the ones called ‘climate models’, and the even bigger ones used in numerical weather prediction – are clearly constructed by physicists, not mathematicians. But they all work the same way: a value of X goes in and a new value X comes out. The mechanism is exactly the same as in the simple maps deﬁned above, even if X might have over 10,000,000 components. Parameters and model structure The rules that deﬁne the maps above each involve numbers other than the state, numbers like four and onehalf. These numbers are called parameters. While X changes with time, parameters remain ﬁxed. It is sometimes useful to contrast the properties of time series generated using different parameter values. So instead of 44 deﬁning the map with a particular parameter value, like 4, maps are usually deﬁned using a symbol for the parameter, say α. We can then contrast the behaviour of the map at α equals 4 with that at α = 2, or α = 3.569945, for example. Greek symbols are often used to clearly distinguish parameters from state variables. Rewriting the Full Logistic Map with a parameter yields one of the most famous systems of nonlinear dynamics: the Logistic Map: Subtract X2 from X, then multiply by α and take the result as the new X. Attractors Recall the Quartering Map, noting that after one iteration every point between zero and one will be between zero and onequarter. Since all the points between zero and onequarter are also between zero and one, none of these points can ever escape to values greater than one or less than zero. Dynamical systems in which, on average, line segments (or in higher dimensions, areas or volumes) shrink are called dissipative. Whenever a dissipative map translates a volume of state space completely inside itself, we know immediately that an attractor exists without knowing what it looks like. 45 Chaos in context: determinism, randomness, and noise In physical models, parameters are used to represent things like the temperature at which water boils, or the mass of the Earth, or the speed of light, or even the speed with which ice ‘falls’ in the upper atmosphere. Statisticians often dismiss the distinction between the parameter and the state, while physicists tend to give parameters special status. Applied mathematicians, as it turns out, often force parameters towards the inﬁnitely large or the inﬁnitesimally small; it is easier, for example, to study the ﬂow of air over an inﬁnitely long wing. Once again, these different points of view each make sense in context. Do we require an exact solution to an approximate question, or an approximate answer to a particular question? In nonlinear systems, these can be very different things. Chaos Whenever α is less than four we can prove that the Logistic Map has an attractor by looking at what happens to all the points between zero and one. The largest new value of X we can get is the iteration of X equals onehalf. (Can you see this in Figure 8?) This largest value is α/4, and as long as α is less than four this largest value is less than one. That means every point between zero and one iterates to a point between zero and α/4 and is conﬁned there forever. So the system must have an attractor. For small values of α the point X equals zero is the attractor, just like in the Quartering Map. But if α is greater than one, then any value of X near zero will move away and the attractor is elsewhere. This is an example of a nonconstructive proof: we can prove that an attractor exists but, frustratingly, the proof does not tell us how to ﬁnd it nor give any hint of its properties! Multiple time series of the Logistic Map for each of four different values of α are shown in Figure 9. In each panel, we start with 512 points taken at random between zero and one. At each step we move the entire ensemble of points forward in time. In the ﬁrst step we see that all remain greater than zero, yet move away from X equals one never to return: we have an attractor. In (a) we see them all collapsing onto the period one loop; in (b) onto one of the two points in the period two loop; in (c) onto one of the four points of the period four loop. In (d), we can see that they are collapsing, but it is not clear what the period is. To make the dynamics more plainly visible, one member of our ensemble is chosen at random in the middle of the graph, and the points on its trajectory are joined by a line from that point forward. The period one loop (a) appears as a straight line, while (b) and (c) show the trajectories alternating between two or four points, respectively. While (d) ﬁrst looks like a period four loop as well, but a closer look shows that there are many more than four options, and that while there is regularity in the order in which the bands of points are visited, no simple periodicity is visible. 46 To get a different picture of the same phenomena, we can examine many different initial conditions and different values for α at the same time, as shown in Figure 13 (page 63). In this threedimensional view, the initial states can be seen randomly scattered on the back left of the box. At each iteration, they move out towards you and the points collapse towards the pattern shown in the previous two ﬁgures. The iterated initial random states are shown after 0, 2, 8, 32, 128, and 512 iterations; it takes some time for the transients to die away, but the familiar patterns can be seen emerging as the states reach the front of the box. We can see now that a dynamical system has three components: the mathematical rule that deﬁnes how to get the next value, the parameter values, and the current state. We can, of course, change any of these things and see what happens, but it is useful to distinguish what type of change we are making. Similarly, we may have insight into the uncertainty in one of these components, and it is in our interest to avoid accounting for uncertainty in one component by falsely attributing it to another. Our physicist may be looking for the ‘True’ model, or only just a useful one. In practice there is an art of ‘tuning’ parameter values. And while nonlinearity requires us to reconsider how we ﬁnd ‘good parameter values’, chaos will force us to reevaluate what we mean by ‘good’. A very small difference in the value of a parameter which has an unnoticeable impact on the quality of a shortterm forecast can alter the shape of an attractor beyond recognition. Systems in which this happens are called structurally unstable. Weather forecasters need not worry about this, but climate modellers must; as Lorenz noted in the 1960s. A great deal of confusion has arisen from the failure to distinguish between uncertainty in the current state, uncertainty in the value of a parameter, and uncertainty regarding the model structure itself. Technically, chaos is a property of a 47 Chaos in context: determinism, randomness, and noise Tuning model parameters and structural stability 9. Each frame shows the evolution of 512 points, initially spread at random between zero and one, as they move forward under the Logistic Map. Each panel shows one of four different values of α, showing the collapse towards (a) a ﬁxed point, (b) a period two loop, (c) a period four loop, and (d) chaos. The solid line starting at time 32 shows the trajectory of one point, in order to make the path on each attractor visible dynamical system with ﬁxed equations (structure) and speciﬁed parameter values, so the uncertainty that chaos acts on is only the uncertainty in the initial state. In practice, these distinctions become blurred and the situation is much more interesting, and confused. Chaos Statistical models of Sun spots Chaos is only found in deterministic systems. But to understand its impact on science we need to view it against the background of traditional stochastic models developed over the past century. Whenever we see something repetitive in nature, periodic motion is one of the ﬁrst hypotheses to be deployed. It can make you famous: Halley’s comet, and the Wolf Sun spot number. In the end, the name often sticks even when we realize that the phenomenon is not really periodic. Wolf guessed that the Sun went through a cycle of about 11 years at a time when he had less than 20 years’ data. Periodicity remains a useful concept even though it is impossible to prove a physical system is periodic regardless of how much data we take. So are the concepts of determinism and chaos. The solar record showed correlations with weather, with economic activity, with human behaviour; even 100 years ago the 11year cycle could be ‘seen’ in tree rings. How could we model the Sun spots cycle? Models of a frictionless pendulum are perfectly periodic, while the solar cycle is not. In the 1920s, the Scottish statistician Udny Yule discovered a new model structure, realizing how to introduce randomness into the model and get more realisticlooking time series behaviour. He likened the observed time series of Sun spots to those from the model of a damped pendulum, a pendulum with friction which would have a free period of about 11 years. If this model pendulum were ‘left alone in a quiet room’, the resulting time series would slowly damp down to nothing. In order to motivate his introduction of random numbers to keep the mathematical model going, Yule extended the 50 analogy with a physical pendulum: ‘Unfortunately, boys with pea shooters get into the room, and pelt the pendulum from all sides at random.’ The resulting models became a mainstay in the statistician’s arsenal. A linear, stochastic mainstay. We will deﬁne the Yule Map: Take α times X plus a random value R to be the new value of X where R is randomly drawn from the standard bellshaped distribution. Yule developed a model similar to the Yule Map that behaved more like the time series of real Sun spots. Cycles in Yule’s improved model differ slightly from one cycle to the next due to the random effects, the details of the pea shooters. In a chaotic model the state of the Sun differs from one cycle to the next. What about predictability? In any chaotic model, almost all nearby initial states will eventually diverge, while in each of Yule’s models even far away initial states would converge, if both experienced the same forcing from the pea shooters. This is an interesting and rather fundamental difference: similar states diverge under deterministic dynamics whereas they converge under linear stochastic dynamics. That does not necessarily make Yule’s model easier to forecast, since we never know the details of the future random forcing, but it changes the way that uncertainty evolves in the system, as shown in 51 Chaos in context: determinism, randomness, and noise So how does this stochastic model differ from a chaotic model? There are two differences that immediately jump out at the mathematician: the ﬁrst is that Yule’s model is stochastic – the rule requires a random number generator, while a chaotic model of the Sun spots would be deterministic by deﬁnition. The second is that Yule’s model is linear. This implies more than simply that we do not multiply components of the state together in the deﬁnition of the map; it also implies that one can combine solutions of the system and get other acceptable solutions, a property called superposition. This very useful property is not present in nonlinear systems. Chaos 10. The evolution of uncertainty under the stochastic Yule Map. Starting as a point at the bottom of the graph, the uncertainty spreads to the left as we move forward in time (upwards) and approaches a constant bellshaped distribution Figure 10. Here an initially small uncertainty, or even an initially zero uncertainty, at the bottom grows wider and moves to the left with each iteration. Note that the uncertainty in the state seems to be approaching a bellshaped distribution, and has more or less stabilized by the time it reaches the top of the graph. Once the uncertainty saturates in a static state, then all predictability is lost; this ﬁnal distribution is called the ‘climate’ of the model. 52 Physical dynamical systems There is no way of proving the correctness of the position of ‘determinism’ or ‘indeterminism’. Only if science were complete or demonstrably impossible could we decide such questions. E. Mach (1905) The time series we want to observe now is the state of the physical system: say, the position of our nine planets relative to the Sun, the number of ﬁsh or grouse. As a shorthand, we will again denote the state of the system as X, while trying to remember that there is a fundamental difference between a modelstate and the True state, if such a thing exists. It is unclear how these concepts stand in relation to each other; as we shall see in Chapter 11, some philosophers have argued that the discovery of chaos implies the real world must have special mathematical properties. Other philosophers, perhaps sometimes the same ones, have argued that the discovery of chaos implies mathematics does not describe the world. Such are philosophers. In any event, we never have access to the True state of a physical system, even if one exists. What we do have are observations, which we will call ‘S’ to distinguish them from the state of the system, X. What is the difference between X and S? The unsung hero of science: noise. Noise is the glue that bonds the experimentalists with the theorists on those occasions when they meet. Noise is also the grease that allows theories to slide easily over awkward facts. 53 Chaos in context: determinism, randomness, and noise There is more to the world than mathematical models. Just about anything we want to measure in the real world, or even just think about observing, can be taken to have come from a physical dynamical system. It cou