Untangling Complex Systems, page 98
The best evaporation-rate is 5. When it is 0, ants have three broad paths to the three
sources of food to follow. When the evaporation-rate is 20, ants can trust just in shreds of
tracks towards the food, and most of them waste time by roving randomly throughout the
entire space available.
13.5. By playing with the number of strategies and the memory size, different dynamics are
observable. A list of results is reported in Table 13.4.
According to the results presented in Table 13.4, a good solution is when both memory
size and number of strategies are fixed to ten. There are overcrowded outcomes, but the num-
ber of people does not exceed too much above sixty, and when there is no overcrowding, the
number of people is close to sixty. The bar is never “desert,” and this is good for the owner.
13.6. The first principle of thermodynamics allows us to write dU = dq + dw. Introducing the second law of thermodynamics, we have dU = TdS − PdV. Since the ideal gases are at constant temperature, their internal energies do not change. Therefore, TdS = PdV. Introducing
500
Untangling Complex Systems
TABLE 13.3
Times Taken by the Ants to Eat the Food as a Function of the Evaporation Rate of Pheromone
Evaporation Rate
Time (ticks)
Average Time
Picture
0
600
790
729
624
775
703.6
5
470
561
493
504
447
495
20
1031
1538
1625
1816
1598
1521.6
TABLE 13.4
Possible Dynamics of Attendance to the El Farol Bar as Functions of the Memory-size and
the Number of Strategies
Memory-size
N° strategies
Dynamics
Attendance
1
1
Fixed point
>70
1
10
Oscillations
] ~85, ~25[
1
20
Oscillations
] ~90, ~10[
5
1
Fixed point or oscillations with small amplitudes
≥70
5
10
Chaotic
] ~70, ~45[
5
20
Chaotic
] ~90, ~20[
10
1
Chaotic (small amplitudes)
≥65
10
10
Chaotic
] ~67, ~50[
10
20
Chaotic
] ~70, ~40[
the equation of state for ideal gases, we obtain that dS = ( nR V ) dV . For the mixing process (I), the total variation of the thermodynamic entropy is:
∆
Vbf
V
S = n
gf
b Rln
+ ngRln
= ( nb + ng) Rln 2
Vbi
V
gi
where the subscripts b and g refer to the black and gray gases, respectively.
How to Untangle Complex Systems?
501
If we consider the information entropy, the uncertainty (H) that we have about a gas con-
tained in a volume Vi is Hi = log 2 Vi. Therefore, in the expansion from Vi to Vf , the increment of uncertainty and hence the variation of the information entropy is ∆ H = log 2 V
(
)
f Vi . It is
evident the strict relation between the two expressions:
(∆ H )( n
b + nr ) Rln 2 = ∆ S
Analogously, for the process of free expansion (∆ H)( nb) Rln 2 = ∆ S.
Appendix A: Numerical Solutions
of Differential Equations
No human investigation can become real science without going through mathematical people.
Leonardo da Vinci (1452–1519 AD)
In addressing the dynamics of a Complex System, the typical approach consists of the following
steps. First, the key variables of the phenomenon are identified, together with the nature of their
interactions. Second, differential equations describing the time evolution of the system are for-
mulated. If the system is spatially homogeneous, the differential equations are ordinary (ODEs);
otherwise they are partial (PDEs). Third, the differential equations are solved either analytically,
graphically, or numerically (most non-linear differential equations cannot be solved analytically).
Before the advent of the computer, numerical methods were impractical because they required an
exhausting work of hand-calculation. With the implementation of always faster computers, all that
has changed. Finally, after determining the fixed points, the stability properties of the steady state’s
solutions are probed.
Differential equations such as [A.1] are rules describing the time evolution of X as a function of X.
dX
= f ( X ) [A.1]
dt
If only first order derivatives are involved, equation [A.1] is a first order differential equation. If
the function f ( X ) is continuous and smooth, i.e . , differentiable in every point, then the solution to equation [A.1] exists and is unique.
In this Appendix A, a few methods of solving numerically differential equations, (such as [A.1]), are presented.
A.1 EULER’S METHOD
The most basic explicit method for the numerical integration of ordinary differential equations
is Euler’s Method. It is named after Leonhard Euler, a Swiss mathematician, and physicist of the
eighteenth century. Euler’s Method converts the rule [A.1] into values for X. Let us suppose that
initially, i.e . , for t = t ,
. The derivative is equal to dX
( ) =
0 X = X 0
f ( X ). Euler’s method assumes
dt t
0
0
that the derivative dX
( )
dt
is constant over a certain time interval Δ t. Therefore, the next value for X
t 0
will be given by
dX
X ( t 0 + t
∆ = t 1) = X 1 = X 0 + f ( X 0) t
∆ = X 0 +
t
∆ [A.2]
dt t 0
503
504
Appendix A
X
X′1
X 1
X 0
t
t 0
t 1
t′1
FIGURE A.1 Description of the function X( t) by the Euler’s method. The solutions shown as black circles are obtained by selecting a longer ∆ t than that chosen for the solutions represented as black squares.
Actually, the derivative f ( X ) is constantly changing, but the assumption that it is constant over the time interval ∆ t becomes less and less wrong, as ∆ t becomes closer and closer to zero, i.e.,
infinitesimal. At time t , in location, the derivative is now equal to f ( X . We step forward to 1
X 1
1)
X 2 = X ( t 1 + t
∆ ) = X 1 + f ( X 1) t
∆ , and then we iterate the procedure:
1. We calculate the derivative in Xn
2. We use the rate of change to determine the next value Xn+1:
Xn+1 = X ( tn + t
∆ ) = Xn + f ( Xn) t
∆ [A.3]
until we have a satisfactory number of solutions.
A visualization of the possible solutions achievable by Euler’s Method is shown in Figure A.1 where a non-linear function X( t) is plotted as a continuous black curve. The squared and the circle dots
represent the solutions of the differential equation [A.1] obtained by applying Euler’s Method with
two different time intervals. The black circles are obtained with a longer ∆ t . It is evident that the
shorter the ∆ t , the more accurate the description of the rule.
A.2 ALTERNATIVE METHODS
An improvement on the Euler’s Method was proposed by the German mathematicians Carl Runge
and Martin Kutta at the beginning of the twentieth century. The Runge-Kutta Method samples the
rate of change at several points along the interval Δ t and averages them. In particular, the fourth-
order Runge-Kutta Method, implemented in MATLAB through the command ode45, requires cal-
culating the following four amounts
k 1 = f ( Xn) t
∆
1
k 2 = f Xn + k 1 t
∆
2
[A.4]
1
k 3 = f Xn + k 2 t
∆
2
k 4 = f ( Xn + k 3)∆ t
Appendix A
505
X
X 0
t
t 0
FIGURE A.2 The gray squares represent the numerical solution of the function depicted by a continuous
black curve.1 They describe the curve much better than the numerical results obtained by the Euler’s Method (see Figure A.1).
to find Xn+1 from Xn. In fact,
1
X 1
( 1 2 2 2 3 4)
n+ = X n +
k + k + k + k [A.5]
6
This method gives much more accurate results than Euler’s Method as it can be easily inferred by
looking at Figure A.2.
Some differential equations may be more critical, and they may require an adaptive step size.
In fact, there are methods where Δ t is automatically adjusted on the fly: Δ t is made small where
the derivative changes rapidly and conversely, it is made large when the derivative changes slowly.
A.3 HINTS FOR FURTHER READINGS
More detailed information about differential equations and their numerical solutions can be found
in many textbooks. See, for example, the book Differential Equations by Blanchard et al. (2006)
and the book by Morton and Mayers (2005) for the numerical solution of partial differential
equations.
1 The syntax for the application of fourth-order Runge-Kutta method in MATLAB is the following one: function dy = heating(t,y)
dy = zeros(1,1);
dy = 0.2*(20−y);
Then, you must paste in the Command Window the following string:
[t,y]
= ode45(‘heating’, [0 12], 5);
Appendix B: The Maximum
Entropy Method
The real voyage of discovery consists, not in seeking new landscapes, but in having new eyes.
Marcel Proust (1871–1922 AD)
The Galilean scientific method is grounded on the collection of experimental data, their analy-
sis, and interpretation. In the time-resolved spectroscopies, the time-evolving signal of a sample
recorded upon pulse-excitation is a convolution of the pulse shape of the excitation (IRF) with the
response of the molecular system ( N( t)) (see Figure B.1).
After deconvolution, the experimental data should be expressed as a weighted infinite sum of
exponentials:
∞
N t
α τ e− t
( )
/
=
( ) τ d
∫
τ [B.1]
0
Our computational task is to determine the spectrum of α( τ) values having inevitably noisy data
available. Mathematically, α( τ) is the inverse Laplace transform of the measured signal decon-
volved from the IRF. Although deconvolution is well-conditioned and relatively stable, inverting
the Laplace transform is ill-conditioned for a noisy signal. This statement means that small errors
in the measurement of R( t) can lead to huge errors in the reconstruction of α( τ). In other words, the ill-conditioning leads to a multiplicity of allowable solutions. The ensemble of all possible shapes of
α( τ) is displayed as a rectangle labeled as set A in Figure B.2. Among all the elements of set A, only a subset of them agrees with the noisy data set shown on the left of Figure B.2. It is the set labeled as B, confined inside the dotted boundaries. Some of these spectra of set B are devoid of physical
meaning (e.g., corresponding to negative concentrations). Therefore, set B is partitioned into two
subsets: subset B” contains the unphysical spectra α( τ) to be rejected, whereas subset B’ contains the feasible spectra α( τ). All the elements of subset B’ fit the data and are physically allowable. Among them, we need to choose just one element. This operation is possible by maximizing some function
of the spectra, h[ α( τ)]. The function h[ α( τ)] should be chosen to introduce the fewest artifacts. For this purpose, we use the Maximum Entropy Method (MEM).
The roots of the Maximum Entropy Method reside in the probability theory. Statisticians are
divided into two schools of thought regarding the interpretation of probability: the “classical,”
“ frequentist,” or “objective” school instituted by Fisher in the 1940s, and the “subjective” or
“Bayesian” school, named after Thomas Bayes who proposed it around 1750 (Brochon 1994). The
“objective” school of thought regards the probability of an event as an objective property of that
event. It is possible to measure that probability by determining the frequency of positive events in
an infinite number of trials. In calculating a probability distribution, the “objectivists” believe that
they are making predictions which are in principle verifiable, just as they were dealing with deter-
ministic events of classical mechanics. The question that an “objectivist” raises when he judges a
probability distribution p( x) is: “Does it correctly describe the observable fluctuations of x?”
The “subjective” school of thought regards the probability as personal degree of belief: the prob-
ability of an event is merely a formal expression of our expectation that the event will or did occur,
507
508
Appendix B
Object of enquiry
Instrument
Experimental result
IRF
R( t)
a.u.
0
Time
R( t) = ( IRF ) ⊗ ( N( t))
Instrumental
True
response
value
function
FIGURE B.1 Schematic representation of the steps of a time-resolved measurement. The sample is the object
of inquiry; the instrument allows to collect a time-resolved signal R( t) that is a convolution of the Instrumental Response Function (IRF) and the molecular kinetics N( t). The analysis of R( t) is carried out by the operator with the help of some computational tools.
Set A
Set B
IRF
Inverse
R( t)
Laplace
transform
Subset B’
a.u.
Subset B”
0
Time
FIGURE B.2 Partition of all possible spectra α( τ) contained in set A, into two ensembles: set B containing all the spectra agreeing with the experimental data and set not B ( B) containing the remaining elements. Set
B is further partitioned into two subsets: B’ containing the spectra with physical meaning and B” containing
all the spectra devoid of physical significance.
based on whatever information is available. The test of a good subjective probability distribution
p( x) is: “Does it correctly represent our state of knowledge as to the value of x?” (Jaynes 1957). The Bayesian view of probability is the source of MEM.
The key aim of scientific data analysis is to determine the probability density of h[ α( τ)] given some data D, i.e., the conditional probability Pr( h D). According to the Bayes’ theorem, the probability of D and h, Pr( h, D) is given by
Pr( ,
h D) = Pr( )
h Pr D
( h) = Pr( D)Pr h
( D ) [B.2]
Appendix B
509
The Bayes’ theorem can be rearranged in the following form:
Pr h
( ) Pr( D h)
Pr( h D) =
[B.3]
Pr D
( )
The term Pr( D h) is the “likelihood:” it is the probability of obtaining some data D if the true spectrum h is known. In general, the likelihood expresses the instrumental response, its resolution and
uncertainty. In the case of Gaussian noise statistics, the likelihood is
−1 2
χ
e 2
Pr( D h) =
[B.4]
Zl
with
1 2 1
T
2
χ =
D −
−
(
R )
h (σ )( D −
)
Rh [B.5]
2
2
where:
Rh is the calculated data from the spectrum h;
σ−2 is the covariance matrix for the data;
Z is the normalization factor.
l
The term Pr( D) of equation [B.3] is the “plausibility” of the data based on our prior knowledge of
the system. If the prior knowledge of the system maintains constant, this term is like a normaliza-
tion factor ( Z ). Otherwise, it can be used to discriminate between models in which the data would
D
have a different probability of occurring, for example, between a set of discrete or continuous dis-
