Instruction:
What is the probability of being 1 unit to the west of our starting point at time 3, that is,
atposition −1?

Response: 18 An introduction to stochastic processes 18.1 Introduction The purpose of this chapter is to provide background for the remaining chapters in the book. We make much use of the concept of conditioning, so the reader may wish to review Sections A.2 and A.8 of Appendix A. A stochastic process is the tool used to model a quantity varying randomly in time. The following are the essential ingredients. We have an index setT, which gives the points of time that we are interested in. Normally,T will either be the nonnegative integers, 0,1,2,… (discretetime) orthe whole nonnegative line[0,∞) (continuous time).Inboth cases, we will sometimes have a maximum time horizon that we are interested in, in which case,∞ will be replaced by a finiteN). We also need a sample space with a probability measureP, and for each t in T, a random variableX t defined on this space. The random variableXt gives the value at timet, of the random quantity that we are trying to model. Astochasticprocess can then be defined formally as a collection of random variablesXt defined for eachti nas e tT. Wewillillustratebrieflybyconsideringthepriceofacertainstock.Anyonewhohasbeen involved with the stock market can attest that this is indeed a quantity that varies in time, and is subject to all kinds of random influences. Let time be discrete and refer to days. Suppose the stock is selling for 100 per share now, and we know that each day it will either increase by 20%or decrease by 20%. Therefore,X 0 takes a value of 100 with probability 1, whileX1 will take a value of either 80 or 120,X2 will take one of three possible values, 144, 96, or 64, and so on. The details are shown in Figure 18.1. Of course, to complete the model, we have to specify probabilities. We will not do this quite yet, as we want to first discuss the complications that arise. The first principle to observe is that we are not just interested in the distribution of each X t but also in all the possible joint distributions. To illustrate, suppose we want to know the FundamentalsofActuarialMathematics , Third Edition. S. David Promislow. © 2015 John Wiley & Sons, Ltd. Published 2015 by John Wiley & Sons, Ltd. Companion website: http://www.wiley.com/go/actuarial 282 AN INTRODUCTION TO STOCHASTIC PROCESSES 100 120 80 144 96 64 172.20 115.20 76.80 51.20 Figure 18.1 Evolution of stock price probabilitythatthestockwillbepricedat115.20attime3.FromFigure18.1,wecanidentify three mutually exclusive ‘path segments’ leading from 100 at time 0 to 115.20 at time 3. The desired probability can then be written as P(X 0 = 100,X1 = 120,X2 = 144,X3 = 115.20)+P(X0 = 100,X1 = 120,X2 = 96, X3 = 115.20)+P(X0 = 100,X1 = 80,X2 = 96,X3 = 115.20). Many of the questions we ask of stochastic processes are, like the above, concerned with realizations.A realization(sometimescalled samplepath orscenario)ofastochasticprocess is a function defined on the index setT, sendingtto xt, which represents a possible outcome of the process, namely that in which the random variableXt takes the valuext for alltin T. For example, in our example above, the realizationxt = 100×1.2t represents the outcome whereby the stock continually moves upward. It is often useful to view a stochastic process as a model for assigning probabilities to realizations. A technical difficulty arises, however. NormallyT is infinite, andXt takes at leasttwovalues,exceptpossiblyfor X0.Thismeansthereare uncountablymanyrealizations, and so each single realization will normally have probability 0. We must therefore deal with infinite sets of realizations, as these can have positive probability. (This is exactly analogous to the situation with any continuous random variableX where we know that the probability that Xtakes a particular value ofxis always 0, but we are interested in the probability thatX takes values in some infinite set.) Note that in our stock example, an event such as (X 0 = 100,X1 = 120,X2 = 144, X3 = 115.20), which we referred to above, is not by itself a realization (unless our index set T were just {0,1,2,3}). Rather, it is an infinite set of realizations, consisting of all those with the given value ofxt for t= 0,1,2,3, and including all possible values fort> 3. MARKOV CHAINS 283 The reader should realize at this point that if the index setT is in fact finite, then a stochastic process is formally nothing more than a multi-dimensionaljointdistribution. 18.2 Markov chains 18.2.1 Definitions Throughout this section, we will assume that our index setT is the nonnegative integers and that eachXt is discrete. Toanswerrelevantquestionsaboutastochasticprocess,wewanttobeabletocomputeall finite joint distributions. Normally, however, we do not do so directly but deduce these joint distributionsthroughsomeotherinformationthatwecandetermineaboutourmodel.Suppose we are at timek, and we want to predict what will happen to our quantity at timek+1. This canbequitecomplicatedasitcoulddependontheentirepasthistoryofwhathappened upto timek.Inmanyapplications,thisissimplified,sincetheonlyrelevantpartofthispasthistory forthispredictionistheactualvalueattime k,andwegetnofurtherinformationfromlooking at the values before that time. The following formulates this precisely. Definition 18.1 A discrete-time stochastic process with discrete random variables is called a Markov chain, if given any finite sequencex 0,x1,x2,…,xk+1, wherexi is a possible value of Xi, P(Xk+1 = xk+1|Xk = xk,Xk−1 = xk−1,…,X0 = x0) = P(Xk+1 = xk+1|Xk = xk). (18.1) To illustrate, consider the stock example given above. Suppose that we decide that each daythestockwillmoveupwithprobabilityof2 ∕3andmovedownwithprobability1∕3.This isclearlyaMarkovchainsinceboththelefthandsideandrighthandsideof(18.1)are2 ∕3if xk+1 = 1.2xk,1 ∕3i fxk+1 = 0.8xk, and 0 in all other cases. Suppose, however, that we decide that these probabilities will hold only when the stock has made two different movements on the previous two days. On the other hand, we decide that if the stock moves up two days in a row, it signifies a trend, and the probability of an upward move on the next day changes to 3∕4, while if the the stock moves down two days in a row, it signifies a pessimistic attitude, and the probability of an upward move on the next day is only 3∕5. This would no longer be a Markov chain, since the probability ofX k+1 is clearly influenced by the values ofXk−2 and Xk−1 as well as that ofXk. For example, takingk= 3, P[X4 = 92.16|X3 = 115.20,X2 = 96,X1 = 80] = 1 4, but P[X4 = 92.16|X3 = 115.20,X2 = 96,X1 = 120] = 1 3. In a Markov chain, these would both have to equal the same number, namelyP[X4 = 92.16|X3 = 115.20]. The important feature of the Markov property is that it allows us to compute all relevant probabilities once we know the probabilities of each ‘branch’ of the tree-like structure, as we have drawn in Figure 18.1. To elaborate, we introduce some fundamental notation. 284 AN INTRODUCTION TO STOCHASTIC PROCESSES For any integersk≤ n,l e t pxy(k,n) = P(Xn = y|Xk = x). (18.2) The probability of any branch is of the formpxy(k,k+1) and we can multiply the proba- bilities of each branch to get the probability of any path. Finally, we can compute the general probability p xy(k,k+n) by adding the probabilities on all paths that lead from a value ofxat timekto a value ofyat timen. Considerourstockexamplewiththe(2 ∕3,1∕3)probabilitiesoftherespectiveupordown move. We can now easily answer our question, and find the probability that the stock price is 115.20 at time 3, that isp 100,115.20(0,3) There are 3 paths leading from the starting value of 100 to the value of 115.20 at time 3. Each of them consists of two up moves and one downmove,andwillhaveprobabilityequalto 2 3 × 2 3 × 1 3 = 4 27,andsothetotalprobabilityin question is 12∕27. Inmanycommonapplications(asinthisone),thereisthefurthersimplifyingfeaturethat px,y(k,k+1) is independent ofk, and can be denoted by justpxy. In this case, we say that we havestationarytransitionprobabilities.(Thisfeatureisalsoreferredtoas timehomogeneity.) In connection with stochastic processes, the wordstationary can be thought of as referring to a process ‘without a watch’. In the present context, it means that whenever the evolving quantity takes a value ofx, the probability that it takes a value ofyat the next stage is always the same, regardless of the particular time. 18.2.2 Examples WenowlookatsomeotherexamplesofMarkovchains.Oneofthemostfamousisthe random walk. An indecisive person goes for a walk but cannot decide whether to go east or west. A coin is flipped and the person goes 1 unit east if a head comes up or 1 unit west if a tail comesup.Aftereachmove,thecoinisflippedagainandtheprocedurerepeated.Supposethe probability of a head coming up isp. LettingX k refer to his position east of the starting point at timek, we have the process shown in Figure 18.2. Thisissimilartothestockquestionaskedabove.Wehavethreepossiblepath segmentsleadingto −1attime3,eachwithprobability p(1−p)2,sotheansweris3p(1 −p)2. Thissamemodelappliestomanysituations.Imagineagamblerrepeatedlyplayingagame withanevenmoneypayoff,suchasbettingonblackatroulette.Inanysingleplay,thisperson eitherwins1unitwithprobability porloses1unitwithprobability1 −p.IfweletX k denote the total winnings afterk plays (a negative amount signifying a loss), then we have exactly the random walk process as described above. Many of the processes that we will deal with can be viewed in this gambling context. It will be convenient to make a slight adjustment. Instead of keeping track of the amount won or lost, we will keep track of the total fortune of the gambler starting from an initial fortune of u. This is just a matter of addinguto each entry. For example, if the gambler starts play with a fortune of 10 units, then the diagram of the process would be as shown in Figure 18.3. Considerasituationwherethewagerateachstageismorecomplicatedthanasimpleeven moneybet.Supposethatthereturntothegambleronthisbetissomediscreterandomvariable G. (In the case above,Gsimply took the value 1 with probabilitypand −1 with probability 1−p.)Thegambler startswithafortuneof u,andwewanttoconsiderthestochasticprocess MARKOV CHAINS 285 3 1 1 –1 0 2 –1 –3 –2 0 Figure 18.2 Random walk 1010 11 13 8 9 12 11 9 7 Figure 18.3 Gambler’s fortune 286 AN INTRODUCTION TO STOCHASTIC PROCESSES Un which equals the fortune of the gambler at timen. LettingGk be the return at timek,w e can write Un = u+G1 +G2 +⋯+Gn, (18.3) where theGis are independent and each distributed asG. This is clearly a Markov chain with transition probabilities given by px,y(k,k+1) = P(G= y−x), and since the right-hand side is independent ofk, we see also that this process has stationary transition probabilities. Note that it would still be a Markov chain, but not necessarily a stationary one, if the gambler were to change the bet at various times, as long as this was done independently of thepasthistoryofwinnings.Anexampleofthiswouldbeforthegamblertodecidethatafter fiveturnsattheroulettetable,regardlessofwhathappens,hewillswitchtoblackjack.Inthis case,thedistributionsofthevarious G n arenotallthesame.If,however,thegamblerdecided that he would play roulette until he had five consecutive losses and then switch to blackjack, the resulting process wouldnothave the Markov property. We return to the process given by (18.3) in Chapter 23 where it will play an important role in modelling the surplus of an insurer. 18.3 Martingales Inthissection,webrieflyintroduceanotherimportantclassofstochasticprocesses,whichwe will make much use of later. Definition 18.2 A discrete-time stochastic process is amartingaleif E(X k+1|Xk = xk,Xk−1 = xk−1,…,X0 = x0) = xk, forall kandvalues x0,x1,…,xk.Inotherwords,atanytime,theexpectedvalueofourquantity at the next time period is exactly what it is now. Note that in the case of a Markov process, the requirement above simplifies to E(Xk+1|Xk = xk) = xk. Is our stock price process with the 2∕3 and 1∕3 probabilities a martingale? The answer is no, since E(Xk+1|Xk = x) = 2 3 ×1.2x+ 1 3 ×0.8x= 3.2x 3 > x. A process like this, for which the expectation of the future random variable is always greater than or equal to the present one, is known as asubmartingale. It would only be a martingaleifthe2∕3werereplacedby1∕2.Indeed,wewouldnotexpectastochasticprocess for stock prices to be a martingale, since it would mean that there was no tendency for the pricetoincrease.However,thereasonforpurchasingstockinthefirstplace,giventheinherent risk, is the expectation of capital gains as the stock increases in value. FINITE-STATE MARKOV CHAINS 287 What about the process given by (18.3)? We have E(Uk+1|Uk = x) = E(Uk +Gk+1|Uk = x) = x+E(G). This process will be a martingale if and only ifE(G) = 0, that is, if and only if the bet made by the gambler is a fair bet. The martingale concept was in fact introduced originally as the model of a fair game. In the usual casino games,E(G) < 0, and we would not have a martingale. Suchaprocess, wheretheexpected valueofthefuturerandom variable isalways lessthanorequaltothepresentone,isknownasa supermartingale.Amartingaleistherefore both a supermartingale and a submartingale. (To remember the terminology, note that the modifier applies to thecurrentvalue. So asubmartingale means that at any time, the current value of the process isunderthe expectation of the future value.) 18.4 Finite-state Markov chains Sometimes, it is convenient to relabel with integers the possible values that can be taken by therandom variablesinaMarkovchain.Werefertotheseintegersasthe statesofthesystem and say that the system is in statei at timek if X k takes the valuei. In fact we often just take N, the number of the state, as the underlying random variable in place. ofX, so that our fundamental probabilitiespij(k,n) will be the probability of being in statej at timen given that the process was in statei at timek. In this section we consider Markov chains with a finite number of states and develop matrix methods for investigating their properties. In this chapter we confine our attention to the stationary case. Nonstationary chains are needed in Chapter 19 and will be introduced at the beginning of that chapter. Starredsectionscontainsomewhatmoreadvancedmaterialandthesecanbeomittedatfirst reading (although we do refer back to one of the results in one of the Chapter 23 examples). 18.4.1 The transition matrix Suppose we have a stationary Markov chain withNstates. We will number them {0,…,N− 1}. (Some authors use the numbers 1 to N instead). By the stationary condition, the quantity p ij(k,k+1) the probability of moving from stateito statejin one step, is, independent ofk, and we can write it as justpij. The matrixP, with entries ofpij in theith row andjcolumn is known as thetransitionmatrix of the chain. Note that the only condition required for anN×N matrix to be the transition matrix of some Markov chain is that all entries are nonnegative, and each row sums to 1. Remark The reader should keep in mind that we have started the indexing at 0, so the top row (or left column) of the matrix will be considered as row (or column) number 0,not number 1. Example 18.1 A box contains two balls, either of which can be red or yellow. At each stage,aballischosenatrandomandreplacedwithaballoftheoppositecolour.Let X n bethe number of red balls in the box at timen. Find the transition matrix. Solution. Herewehaveathree-stateMarkovchain.Wewillnumberthestatesbythenumber ofredballsinthebox.Ifthereare0or2redballs,wearesuretomovetostate1,whileifthere 288 AN INTRODUCTION TO STOCHASTIC PROCESSES is 1 red ball, we move to either state 0 or 2, with equal probability. Therefore the transition matrix is P = ⎛ ⎜ ⎜ ⎝ 010 1∕201 ∕2 010 ⎞ ⎟ ⎟ ⎠ . 18.4.2 Multi-period transitions We now wish to calculate the general probabilitiespij(k,k+m) which by stationarity will be equal topij(0,m). We start by calculating this form= 2. Suppose we move from statei to statekattime1andthenfromstate ktostate jtime2.Weknowfromourpreviousdiscussion onMarkovchainsthattheprobabilityofthistwo-stepmoveoccurringisjust pikpkj.Summing this over all statesk, we get the probability of moving fromitojafter two periods as N∑ k=1 pikpkj, (18.4) which, by the ordinary rules of matrix multiplication, is justP2(i,j)t h e(i,j)th entry of P×P = P2. The same argument can be repeated to show the important fact that in a stationary chain, pij(0,m) is equal to the entry in rowiand columnjofPm (18.5) (Notethat P0 isjusttheidentitymatrix,usuallydenotedby I,withentriesof1onthemain diagonal, and entries of 0 elsewhere.) 18.4.3 Distributions Given a finite-state Markov chain, what is the distribution ofXn? This is a basic question that we ask about any stochastic process. The distribution can be given as anN-dimensional vector𝜋n, whoseith entry, denoted by𝜋n(i), equalsP(Xn = i). This will depend of course on 𝜋0,thevectorgivingtheinitialdistributionattime0.(Incertainapplications,wemightknow the value ofX0, in which case,𝜋0 will simply be a vector with a single entry of 1, and other entries equal to 0.) Starting withn= 1, we calculate P(X1 = j) = ∑ i 𝜋0(i)pij, which is just thejth entry of the vector obtained by multiplying the vector𝜋0 (viewed as a 1×N matrix) on the right byP. The same argument holds for anyn, and we can conclude 𝜋n = 𝜋0Pn. (18.6) Example18.2 InExample18.1,supposewestartwithauniformdistribution,thatis,there is a 1∕3 chance thatX0 takes each of the values 0,1,2. What is the probability thatX101 = 1? FINITE-STATE MARKOV CHAINS 289 Solution. Computing large powers of matrices is often done by calculating eigenvalues, but we can avoid that here. After a couple of multiplications, we see thatP3 = P. Therefore P5 = P3P2 = PP2 = P, and similarly,anyoddpower ofP equalsP. From (18.6), 𝜋101 = (1∕3,1∕3,1∕3) ⎛ ⎜ ⎜ ⎝ 010 1∕201 ∕2 010 ⎞ ⎟ ⎟ ⎠ = (1∕6,2∕3,1∕6). The probability thatX 101 equals 1 is 2∕3. Remark Somewritersdefinethetransitionmatrix Pbytakingtheentryinrow iandcolumn jas the probability of moving from statejto statei. In this case, columns, rather than rows, add to 1. The vectors𝜋n are written as column vectors and in (18.6), we multiply𝜋0 on the leftto get𝜋n. ∗18.4.4 Limiting distributions The distributions𝜋n will normally continue to change with time, but in many applications, we would like to show that there is some limiting distribution𝜋 which is independent of the initial state at time 0. That is, for any initial statej, and all statesi, the resulting probabilities 𝜋n(i)willconvergeto 𝜋(i)as napproaches∞.Thiswouldenableustopredictwithreasonable accuracytheprobabilitiesofbeinginvariousstates,providedtheprocesshasbeencontinuing forasufficientlylongtime.Weconsiderheretheproblemoffindingthislimitingdistribution, provided it exists. The last provision is necessary, as indicated by the following examples where a limiting distribution does not exist. Consider the chain of Example 18.1. If the process is in state 0 or 2, it will move to state 1 in one transition. If it is in state 1, it will move to state 0 or 2 in one transition. So if we start, say in state 0, the process is sure to be in state 0 or 2 at even times, and in state 1 at odd times. Clearly, no limiting distribution can exist. This is an example of what is known as a periodicchainofperiod2.Ingeneral,thismeanswecandividetheset Sofallstatesintotwo disjoint subsets,S 1 and S2, such that in any one transition, all states inS1 move toS2 and all states inS2 move toS1. More generally, we could have chains of perioddwhere we can find d pairwise disjoint subsets, such that from each subset, we cycle through the other sets in a fixed order, and return to the original set afterdtransitions. Another example where a limiting distribution will not exist is the chain with transition matrix P = ⎛ ⎜ ⎜ ⎝ 1∕21 ∕20 1∕21 ∕20 00 1 ⎞ ⎟ ⎟ ⎠ . In this case, if we start in state 2 at time 0, then we remain in state 2 forever. If we start in state 0 or 1, we will reach a limiting distribution, which is to be in state 0 or 1 with equal probabilities.(In fact, this limiting distribution is achieved exactly at time 1). The problem hereisthatthelimitingdistributionsvarysaccordingtotheinitialstate.Thisisanexampleof what is known as areduciblechain. This means we can divide the setSof all states into two nonempty disjoint subsetsS 1 and S2 such that any state inS1 transfers to another state inS1 290 AN INTRODUCTION TO STOCHASTIC PROCESSES after one transition and any state inS2 transfers to another state inS2 after one transition. We can really consider such a Markov chain as two separate chains, one comprising the states in S 1 and the other comprising the states inS2. A simple condition that ensures the existence of a limiting distribution is that there is a positive integern such that all entries ofPn are positive. That is, given any statesi,j (not necessarilydistinct),thereissomechanceofgettingfrom itojinntransitions.Aproofofthis result is beyond the scope of this book. It is, however, not difficult to see that this condition rules out both periodic and reducible chains. See Exercise 19.12. If we know that a limiting distribution𝜋 exists, then there is a straightforward procedure for finding it from the matrixP. We know that𝜋 = lim n→∞ 𝜋n (in the sense of converging at each i as described above). By continuity considerations, we must have𝜋P = limn→∞ 𝜋nP. From (18.6), we know that𝜋nP = 𝜋0PnP = 𝜋0Pn+1 = 𝜋n+1. But clearly𝜋n+1 also converges to𝜋. This establishes that 𝜋 = 𝜋P. (18.7) Note that a solution𝜋 of Equation (18.7) need not be a limiting distribution, since none may exist. It is true that once we have a distribution𝜋 satisfying this equation, we will remain with that distribution forever, but it is possible that from some initial states we will never converge to𝜋. Example 18.1 is an illustration. We have exactly one solution to (18.7), namely 𝜋 = (1∕4,1∕2,1∕4), but as we have shown, we will not approach this for all initial distributions. (It is of interest to note that this is the distribution we would get if we started the process by choosing the colour of each ball randomly.) ∗18.4.5 Recurrent and transient states The states of a stationary Markov chain can be divided into two classes. Given any statej,l e t fj be the probability that, starting in statej, the process will return to that state. Definition 18.3 Statejissaidtobe transientiffj < 1.Inthiscase, thereissomechance that the process will never return. Statejis said to berecurrentif fj = 1. In this case, the process is sure to return. Example 18.3 Classify the states of the Markov chain with transition matrix P = ⎛ ⎜ ⎜ ⎝ 3∕41 ∕40 1∕21 ∕20 1∕403 ∕4 ⎞ ⎟ ⎟ ⎠ . Solution. State2istransient,sincestartingatstate2,thereisapositive chance ofmoving to state 0. From there, only states 0 and 1 can be reached, and there is no return to state 2. Wewillshowthatstates0and1arerecurrent.Supposeweareinstate0.Wefirstobserve thatitiscertainthatwewilleventuallygettostate1.Foranypositiveinteger n,theprobability of staying in state 0 forever is certainly less than the probability of staying in state 0 for the nextntransitions, which is (3∕4) n. As this quantity approaches 0, we see that the probability ofstayinginstate0forever is0,andsowithprobability1wewilleventually move tostate1. Similarly,ifweareinstate1,wearecertaintomovetostate0.Therefore,ifweareinstate0, we are certain to move to state 1 and then back to state 0 Arguing in the same way for state 1, we see that both states are recurrent. FINITE-STATE MARKOV CHAINS 291 This is a fairly simple case, and in general it might not be so easy to classify the states from the matrix. We can, however, adapt the argument in Example 18.3 to prove a general result, which can be of great help in the classification. Definition 18.4 We say a statejis reachablefrom a stateiif there is a positive probability ofeventuallymovingfromstate itoj.Intermsofthematrix P,thisconditioncanbestatedas P n(i,j) > 0f o rs o m en. Theorem 18.1 Ifiisrecurrentandjisreachablefromi,thenjisrecurrent. Proof. Starting in statej, we are certain to eventually reach statei, for if not, then there would be a positive probability of going fromitojand never returning toi, contradicting the fact thatiis recurrent. Now, starting in statei,l e t𝛼be the probability that we will return to state iwithout ever hitting statej. Sincejis reachable fromi, we must have that𝛼< 1. The probability that starting in stateiwe will makenreturn visits toiwithout ever hitting state jis 𝛼n, which approaches 0 asngoes to∞. Therefore, the probability is 0 that, starting ini, we never reach statej, which means that we are in factcertainto eventually reach statej.T o conclude, starting in statej, we are certain to reach stateiand certain to come back to statej from statei, showing that statejis recurrent. Starting in a recurrent state, we must always remain in recurrent states. What happens if we start in a transient state? Theorem 18.2 In a finite-state stationary Markov chain, there is at least one recurrent state.Moreover,startingfromanytransientstate,wemusteventuallyreacharecurrentstate. Proof. Given a transient statej, consider all realizations of the process for which there is at leastoneoccurrenceof j.Theprobabilitythattherewillbeexactlyonesuchoccurrence,given that there is at least one, is 1−f j, the probability of never returning toj. The probability that therewillbeexactlytwosuchoccurrences,giventhatthereisatleastone,isjust fj(1−fj).The process must return once, and then never return again. Continuing, we see that the number of occurrences ofj, less 1, given that there is at least 1, has a geometric distribution.(See Section A.11.3) Now a geometric distribution is a proper frequency distribution with no probabilityofassumingthevalue ∞.Ourconclusionisthattheprobabilityofinfinitelymany occurrences of any transient statej is 0. Consider the event that the chain never visits a recurrent state. If this occurs, since there are only finitely many transient states, one of them must appear infinitely often, but as we have seen the probability of this is 0. This means that with certainty, we must reach a recurrent state. WecanapplythelasttwotheoremstoExample18.3.Onceweseethatstate3istransient, weknowbyTheorem18.2thateitherstate1orstate2isrecurrent,andthenbyTheorem18.1 (or even by symmetry) they both must be recurrent. Note that Theorem 18.2 need not hold for an infinite-state chain. For a trivial example, taketherandomwalkwheretheprobabilityofmovingtotherightis1.Allstatesaretransient. Some transient states are ‘less transitive’ than others. That is, the expected time spent in that state is longer. Indeed, for many applications we may be interested in this expected time spent in a certain transitive statej, that is, the expected number ofnfor whichX n = j. (Thereisnopointinaskingthisquestionforarecurrentstate,since,bydefinition,theprocess 292 AN INTRODUCTION TO STOCHASTIC PROCESSES is in the state for infinitely many values ofn.) Of course, this expectation may depend on the starting statei. Once again this question is of interest only for a transitive starting state i, since for a recurrent starting state, we know by Theorem 18.1 that the answer is 0. We calculate this expectation by the familiar trick of looking atindicatorrandom variables (see the end of Section A.5). Fix such a starting stateiand consider the random variableI n that takes the value 1 ifXn = j,o r0i fX n ≠ j. Then the expected number of visits to statej is just E[∑∞ n=0 In] = ∑∞ n=0 E(In), since A.22 extends to infinite sums for nonnegative random variables. SinceE(In) = pij(0,n), we have Expected visits to statejstarting from statei= ∞∑ n=0 pij(0,n). (18.8) The right hand side of (18.8) may seem quite formidable to calculate. It is possible to use eigenvalues of the transition matrixPto shorten this, (see Section 19.3.4) but there is a much simpler method, based on an idea that is useful in many contexts. Recall the formula for an infinite geometric progression. For a numberx, of absolute value less than 1, (1−x) −1 = 1+x+x2 +⋯. Similarly, it can be shown that for a matrixQ with sufficiently small entries, the matrix I−Qis invertible, and (I−Q)−1 = I+Q+Q2 +⋯. Supposethatthetransientstatesinourmatrixarenumbered0, ….m−1andtake Qtobe them×msubmatrix consisting of the firstmrows and firstmcolumns. Using Theorem 18.1, we can show that fori,jbetween 0 andm−1, the (i,j)th entry ofQn is equal topij(0,n) with the final result that for any two transient statesi,jnot necessarily distinct, Expected visits to statejstarting from statei= the (i,j)th entry of (I−Q)−1 (18.9) Example18.4(Randomwalkwithabsorbingbarriers) Considerarandomwalkonfour consecutive points 1,2,3,4 on a line. Starting at either 2 or 3, the process moves right with probability 2∕3 or left with probability 1∕3. Whenever the process gets to either 1 or 4, it remains there forever. Classify the four states as to recurrent or transitive, and for each pair (i,j)oftransitivestatesfindtheexpectednumberoftimestheprocesswillbein jstartingfrom statei. Solution. Points 2 and 3 are clearly transient. Points 1 and 4 are clearly recurrent, in fact a particular type of recurrent state, known as anabsorbingstate, which means that once there, you never leave. Numbering point 2 as state 0 and point 3 as state 1, for the transient state submatrix Q,w eh a v e Q = ( 02 ∕3 1∕30 ) , I−Q = ( 1 −2∕3 −1∕31 ) ,( I−Q) −1 = ( 9∕76 ∕7 3∕79 ∕7 ) . The final matrix gives the expected number of visits. For example, starting at point 2, the expected number of times the process visits point 3 is 6∕7. POISSON PROCESSES 293 18.5 Introduction to continuous time processes We now turn our attention to processes where the index setT takes all values in the interval [0,∞). We will also allow continuous rather than discrete random variables. One goal is to define continuous-time analogue for the process given in the discrete case by (18.3). In Chapter 23, we apply this to modelling the surplus of an insurer. Another goal is to introduceBrownian motionwhich can be viewed as a type of continuous-time random walk, which we apply in Chapter 20. Notation For continuous-time processes, we will write the time variable in brackets rather than as a subscript. That is, we writeX(t) in place ofX t. We writeX ∼ Y where Xand Y are any two random variables to mean that they have the same distribution. We begin with a few general definitions that capture some of the features we introduced in the discrete time setting. Given a stochastic processX(t) and two timess< t, the random variable X(t)−X(s)i sc a l l e da nincrementof the process, since it gives the increase in the value over the period running from timesto timet. Definition 18.5 We say that the process hasindependentincrements if the increments over disjoint time intervals are independent. This constitutes a strong version of the Markov property. Given timess< t, we can write X(t) = X(s)+X(t)−X(s).Thisshowsthatthevalueof X(t)cancertainlydependonthevalue of X(s), but we get no additional information from looking at times befores, since both X(t)−X(s) andX(s) are independent of what happened in the interval [0,s). Definition 18.6 We say that the process hasstationaryincrements if the distribution of any incrementdependsonlyonthelengthofthetimeintervalandnottheparticularstartingpoint. That is, given anys,t,h> 0,we require that X( s+h)−X(s) ∼ X(t+h)−X(t). This constitutes a strong version of the assumption of stationary transition probabilities that we made for discrete Markov chains. 18.6 Poisson processes Suppose we have a particular ‘event’ that we are interested in, occurring repeatedly and randomly in time. This could be an insurance claim or the arrival of a person at a queue, or a number of other possibilities. Acountingprocess is a stochastic processN(t) that counts the numberofsuch‘events’thathaveoccurreduptotime t.Formally,itisjustanycontinuous-time process that takes nonnegative integer values, and such that all realizations are increasing. A particularrealizationcanthenalwaysbedrawnasanincreasingstepfunction.Wewillalways assume that in any counting process,N(0) = 0. In other words, the counting starts at time 0 294 AN INTRODUCTION TO STOCHASTIC PROCESSES before any events have occurred. A major application in this text will be to insurance claims, which is discussed in Chapter 23. We will confine attention to the particular case of Poisson processes defined as follows. Definition 18.7 A counting processN(t)i sc a l l e daPoisson processwith rate𝜆 if it has stationary and independent increments and if, for allh> 0, N(h) ∼ Poisson(𝜆h). It follows from the stationary increment assumption that, given anyt> 0, the random variableN(t+h)−N(t) ∼ Poisson(𝜆h).Inotherwords,aPoissonprocessissimplyacounting processwithindependentincrements,suchthatthenumberofoccurrencesinanytimeinterval is a Poisson distribution, with parameter proportional to the length of the interval. WhenshouldwechooseaPoissonprocesstomodelacountingsituation?Thereisanother characterization of Poisson processes that gives some insight into answering this question. Suppose we assume that the increments are indeed stationary and independent. The alternate formulation says essentially that we will get a Poisson process if it is ‘highly unlikely’ to have more than one event occurring in a ‘sufficiently small’ time interval. Therefore, if you feel that this is the case for the particular event you are trying to model, you can be justified in choosing the Poisson process. To derive this characterization, we must first give precise meaning to the phrases ‘highly unlikely’ and ‘sufficiently small’. This is done conveniently through the ‘littleo’ notation, which we will now review. We say that a functionf defined on an interval [0,b]i so(h)i f lim h→0 f(h)∕h= 0. This means thatf is getting small rapidly ashgets small, more rapidly thanhitself. For example, f(h) = h2 iso(h),while f(h) = √ hisnot.Weoftenwritethesymbol o(h)forsuchafunction. For example, we would write e𝛽h = 1+𝛽h+o(h), as can be seen from the Taylor series expansion. It is clear that, given two functionsf and g that are botho(h), their sumf +g is o(h), and for any constantc the functioncf is o(h)a s well. We can now state the desired characterization. Theorem 18.3 A counting process that has stationary and independent increments is a Poissonprocesswithrate 𝜆ifandonlyifthefollowinghold: (i) P(N(h) = 1) = 𝜆h+o(h), (ii) P(N(h) ≥ 2) = o(h). Partial proof. One direction is clear. For a Poisson process with rate 𝜆,w eh a v e P[N(h) = 1] = 𝜆he −𝜆h = 𝜆h[1−𝜆h+o(h)] = 𝜆h+o(h). Similarly, we haveP[N(h) = 0] = e−𝜆h = 1−𝜆h+o(h). Therefore,P[N(h) ≥ 2] = 1−P[N(h) = 0]−P[N(h) = 1] = 1−(1− 𝜆h+o(h))−(𝜆h+o(h)) = o(h). The converse is the more difficult part. We assume conditions (i) and (ii) and must show thatN(h)isPoisson.Therearevariousmethodsofdoingthisandwewillnotelaboratefurther. A nice proof can be found in Ross (2010, Theorem 5.1) where this is done by calculating the moment generating function ofN(h). BROWNIAN MOTION 295 18.6.1 Waiting times Givenanycountingprocess, N(t),thereisanassociated waiting-timeprocess,Wn,n= 1,2, …, where Wn is the time between the (n−1)th event and thenth event. We take the 0th event as occurringattime0.So,forexample,ifthefirsteventoccursattime1,thesecondattime1.7, the third at time 2.3, we would haveW 1 = 1, W2 = 0.7, andW3 = 0.6. For some problems, it is more convenient to deal withWn rather thanN(t). Note that the waiting-time process is a discrete-time process (although the index does not refer to times exactly) with continuous random variables, while the counting process is a continuous-time process with discrete random variables. A natural question is to investigate the waiting-time process for a Poisson process, and this is easily answered. Let𝜆 be the rate of the process. Suppose thatW 1 = w1,W2 = w2,…,Wn−1 = wn−1. What is the distribution ofWn?I fs= w1 +w2 +wn−1,t h e(n−1)th occurrence was at times, and in order forWn to be greater than or equal tow, we require that there be no occurrences in the interval (s,s+w], which, by the stationary increment assumption,hasthesameprobabilityasnooccurrencesintheinterval(0, w].Thisprobability isjuste −𝜆w,whichiseasilyrecognizedasthesurvivalfunctionofanexponentialdistribution. We conclude that theWn are independent and each is distributed as Exp(𝜆). Another random variable that is of interest in many applications isTn, the time of thenth occurrence(sometimescalledthe ntharrivaltime).Clearly, Tn = W1 +W2 +⋯+Wn,andit is easy to see using moment generating functions thatTn is distributed as Gamma(n,𝜆). 18.6.2 Nonhomogeneous Poisson processes In many counting processes, the stationarity assumption is not realistic, as we can expect the rateofoccurrence tovarywithtime.Tomodel this,weuseatypeofprocessthatissimilarto a Poisson process, except the rate𝜆 is no longer a constant but rather a function oft. Definition 18.8 A counting process is called anonhomogeneous Poisson processwith intensity function𝜆(t) if it has independent increments and, for allt> 0, (i) P(N(t+h)−N(t) = 1) = 𝜆(t)h+o(h), (ii) P(N(t+h)−N(t) ≥ 2) = o(h). It can then be shown fors< t, the incrementN(t)−N(s) ∼ Poisson(𝜙(s,t)), where 𝜙(s,t) = ∫ t s 𝜆(r)dr. We omit the proof. Note that when𝜆(t) is a constant𝜆, then𝜙(s,t) is just equal to (t−s)𝜆, and we have exactly the same conclusion that we had before in the case of a regular Poisson process. 18.7 Brownian motion 18.7.1 The main definition We now look at another continuous-time stochastic process which has several applications. This is intended as an introduction and we will not supply all proofs. 296 AN INTRODUCTION TO STOCHASTIC PROCESSES A stochastic processX(t),0 ≤ t< ∞, is called aBrownianmotion process with variance parameter 𝜎2,f o rs o m e𝜎>0, if it satisfies the following four conditions: (i) X(0) = 0; (ii) the process has independent and stationary increments; (iii) for eacht> 0,X(t) is normally distributed with mean 0 and variance𝜎2t; (iv) the realizations,t→ xt are continuous functions oft. NotethatonaformalbasisthedefinitionissimilartothatofaPoissonprocess,althoughthe nature of the processes are quite different. Both have independent and stationary increments. In the Poisson case, the increments have a Poisson distribution with expectation proportional tothelengthoftheinterval,andintheBrownianmotioncase,theyhaveanormaldistribution with variance proportional to the length of the interval. In the Poisson case, the realizations are step functions which involve sudden jumps, in contrast to the continuous realizations of Brownian motions. AstandardBrownianmotionisoneinwhich 𝜎 2 = 1.Wewilldenotethisby B(t).Forthe Brownian motionX(t) with variance parameter𝜎2 it is clear that X(t) = 𝜎B(t). 18.7.2 Connection with random walks A Brownian motion can be viewed as acontinuousrandom walk, where moves are made at each instant of time. Consider first our simple random walk of section 18.2.2. We start at 0 on the real line, and each time unit, we move one unit either to the right or left, with equal probability.Letuscomputethedistributionof X(t),ourpositionattime t.Itiseasytoseethat this is related to a binomial distribution. Precisely X(t) ∼ 2Bin(t,1∕2)−t, (18.10) from which it is clear that (see (A.41)) E(X(t) = 2t(1∕2)−t= 0, Var(X (t) = 4t(1∕4) = t. We see that our simple random walk has the mean and variance of condition (iii) in the definition of a standard Brownian motion. Let us now speed up our random walk by making moves more frequently. Instead of movingeachtimeunit,letusmoveevery1 ∕mofatimeunitwhere missomepositiveinteger. Of course, we also want to change the length of each move. One may think that instead of moving1unitateachstep,weshouldnowmove1∕m ofaunit.However,thiswillnotpreserve the variance condition that we want. To do so, we need to make much larger moves. A key fact is that we will need to make the length of each move equal to 1 √ m. So for example, if m= 4,wewouldmakemovesoflength1∕2attimes1∕4,2∕4,3∕4,etc,whileif m= 100,we would make move of length 1∕10 at time 1∕100, 2∕100 etc. LetX(m)(t) denote our position at timet under this new arrangement. It is not hard to see that this will be the same as the BROWNIAN MOTION 297 position at timemtof the simple case with moves of 1 unit each time period, only multiplied by the length of each move, which is 1∕ √ m. From (18.10) X(m)(t) ∼ 2√ m Bin(mt,1∕2)− mt√ m , and we see as above that E(X(m)(t) = 0,Var(X(m)(t) = t. The standard Brownian motion B(t) can be viewed as a limiting case of X(m)(t)a s m approaches ∞. ∗18.7.3 Hitting times Formanystochasticprocessesweareinterestedinthetimethatitfirstreachesacertainpoint. These random times are known ashittingtimes. We will deduce hitting time distributions for Brownian motion, by invoking one of its key properties, known as thesymmetry principle. This simply says that if the process is at a pointb at time s, then at any later timet the distribution must be symmetric aboutb. In other words, given thatB(s) = b, then for any numbersh< k. P(b+h≤ B(t) ≤ b+k) = P(b−k≤ B(t) ≤ b−h) (18.11) This feature is clear for the simple Random walk, where we are just as likely to move to the right as to the left, and therefore is true for Brownian motion by viewing it as a limiting case of random walks. Let B(t) be a standard Brownian motion, and for any pointa,w el e t T a =i n f{t: T(t) = a} To compute the distribution ofTa, we will derive a relationship between the distribution of Ta and distribution ofB(t). Suppose first thata> 0. Then by Equation (A.30) P[B(t) ≥ a] = P[B(t) ≥ a|Ta ≤ t]P[Ta ≤ t] +P[B(t) ≥ a|(Ta > t]P[Ta > t]. We note first that the second term on the right equals 0. Indeed, IfTa > t then the first time the process hitsais after timet, and the value at timetcould not be greater than or equal to a, since if so, by continuity of the realizations, we would have needed to reachasometime before timeT. Consider now the first term. The quantityP(B(t) ≥ a|Ta ≤ t) = 1∕2, sinceTa ≤ t means thattheprocesshit aatsometime spriorto tandbythesymmetryprincipleitisjustaslikely to be below as above. Formally, we are applying (18.11) withh= 0,k=∞ . This enables us to state the desired relationship P(Ta ≤ t) = 2P(B(t) > a). 298 AN INTRODUCTION TO STOCHASTIC PROCESSES SinceB(t)isnormalwithmean0andvariancet,weknowthat P(B(t) > a) = 1−Φ(a∕ √ t) where Φ is the c.d.f of the standard normal. Moreover, fora< 0, the distribution ofTa is, by symmetry, the same as thatT−a since we start the process at 0. Our final conclusion is then P(Ta ≤ t) = 2[1−Φ(|a|∕ √ t)]. Thisenablesustofindthedistributionofanotherrandomvariableofinterest.Let M(t)be the maximum value thatBassumes in the interval [0,t]. This will be of course a nonnegative random variable, since we start at 0. Fora> 0, we use continuity again to see thatM(t) will belargerthanorequalto aifandonlyiftheprocessreached asometimesintheinterval[0, t] which means thatTa ≤ t.S o P(M(t) ≥ a) = 2[1−Φ(|a|∕ √ t)]. This is an intuitively appealing result, since it says that for anya> 0, the probability that a standard Brownian motion takes a value greater thanaat any time before timetis exactly twice the probability that it takes a value greater thanaat timet. ∗18.7.4 Conditional distributions Given two timessandt, what is the conditional distribution ofBt given thatXs took a certain valueb? We consider the two cases. Ifs≤ t,thisisrelativelystraightforward.Wecanimaginetheprocessjuststartingallover again at time s except with a starting value ofbrather than 0. It is then easy to deduce that Ift≥ s,( B(t)|B(s) = b) is normal with meanband variance (t−s). (18.12) What ifs> t? Now we are asking for the distribution of a random variable conditional on future values and the procedure is more complex. Could we perhaps get the same answer except with variance= (s−t)? Obviously, not since this would be false fort= 0w h e r ew e knowbydefinitionthat B t takesthevalue0withcertainty.Itturnsoutthatwehavetomultiply these quantities byt∕s. Wewillworkwiththedensityfunctionsandlet fr standfor fB(r).Nowgiventhat B(s) = b, we will haveB(t) = a provided that the incrementB(s)−B(t) takes a value ofb−a.B y stationarity, this increment is distributed asBs−t. Then ft(a|Xs = b) = ft(a)fs−t(b−a) fs(b) , where we invoke the hypothesis of independent increments for the numerator. Letr= t∕s. Plugginginthevariousnormaldensitiesanddoingsomealgebra,thelefthandsidereducesto Ke x p (−(a−rb)2 r(s−t) ) for some constantKand we can then deduce that in place of (18.12) we have the following. Ift< s,( B(t)|B(s) = b) is normal with meanrband variancer(s−t), wherer= t∕s. NOTES AND REFERENCES 299 18.7.5 Brownian motion with drift ABrownianmotionprocessisamartingale.Wehavenotdefinedthispreciselyforacontinuous time process, but the idea is similar to that of the discrete time case. If a Brownian motion process takes the valuebat some point of time, then the expected value at any future point of time will beb, as the symmetry principle shows. One often wants to model situations where this is not the case and the quantity under study is growing in value. A simple model of this typeisaBrownianmotionwithadriftcoefficient 𝜇.Thisisaprocesswhichstillhasstationary and independent increments,X(t) is still normally distributed with variance𝜎 2t, but now, the mean ofX(t)i s𝜇trather than 0.In other words, on the average the quantity grows as a rate of 𝜇 per time period, so in any time period of lengthh, the quantity can be expected to increase by𝜇h. To state this precisely,X(t) is a Brownian motion with drift coefficient𝜇 and variance parameter 𝜎2 if and only if X(t) ∼ 𝜎B(t)+𝜇t. 18.7.6 Geometric Brownian motion Weareofteninterestedinquantitieswhichgrowataconstant relativeratherthan absoluterate ofgrowth.Anexampleisstockprices.Wemight,forexampleexpectastocktogrowinvalue atarateofsay5%peryear.Apossiblemodelinthiscaseis geometricBrownianmotion .This is a positive valued processX(t) such thatY(t), the logarithm ofX(t)i saB r o w n i a nm o t i o n with drift parameter𝜇 and variance parameter𝜎 2. That is, X(t) = eY(t) = e𝜇t+𝜎B(t). Note that eachX(t) will have a log-normal distribution. When𝜎 = 0,X(t)issimplytheamountofaquantitygrowingexponentiallyintimeatrate 𝜇.SoageometricBrownianmotioncanbeviewedasaprocessthathasanunderlyingpattern ofexponentialgrowth,witharateofgrowththatisnotconstantbutisitselfsubjecttorandom changes as given by a Brownian motion. This is a commonly used model for stock prices in modern finance theory and we will so apply it in Section 20.18. A quantity which is useful in many applications is the conditional expectation ofX(t), giving that the value ofX(s) = afor somes< t. To compute this, we first write X(t) = e Y(t) = eY(s)e(Y(t)−Y(s)) = X(s)e(Y(t)−Y(s)). We know thatY(t)−Y(s) is normal with mean (t−s)𝜇 and variance (t−s)𝜎2.S ou s i n gt h e expression given in Equation (A.58) E(X(t)|X(s) = a) = ae𝜇(t−s)+(t−s)𝜎2∕2. Notes and references Ross (2010) has a good basic introduction to stochastic processes, covering more material than we do here. The section on Poisson processes is based largely on Chapter 5 of that book.Hoel etal.(1972)giveasomewhatmoreadvancedtreatment.KemenyandSnell(1963) 300 AN INTRODUCTION TO STOCHASTIC PROCESSES provide extensive coverage of finite-state Markov chains. A proof for the sufficient condition givenfortheexistenceofalimitingdistributioncanbefoundthereinTheorem4.12.Mikosch (1998) has more material on Brownian motion, with applications to some of the material we discuss in Chapter 20. Exercises 18.1 Acertainstockhasapriceof100attime0.Atanytime k,let Mk denotetheaverage of the prices at times 0,1,…,k. The price at timek+1 will be eitherMk,Mk +20, or Mk −20, each with probability 1∕3. What is the distribution of the stock price at time 2? What is the expected value at time 2? Is this process a Markov process, martingale, submartingale, supermartingale? 18.2 Suppose that rblack balls andrwhite balls are distributed equally among two urns. At each trial, a ball is chosen randomly from each urn and put in the other urn. Let X n be the number of black balls in urn 1 at timen. Describe the transition function for this Markov chain. 18.3 Thepriceofacertainstockiseither9,10or11.Ifthepriceis9onanyday,thenext day’s price will either be 9 or 10 with equal probability. If the price is 11, the next day’s price will be either 11 or 10 with equal probability. If the price is 10, the next day’s price will be either 9, 10, or 11 with equal probability. (a) Model this as a Markov chain and write down the transition matrix. (b) If the price is 9 on Monday, what is the probability that it will be 11 on Friday? ∗(c) Findtheproportionofthetime,inthelongrun,thatthepricewillbeeachofthe three possible values. *18.4 Consider the Markov Chain on states 1 to 5 with transition matrix ⎛ ⎜ ⎜ ⎜ ⎜ ⎝ 2∕30 01 ∕30 01 ∕20 01 ∕2 1∕403 ∕40 0 1∕20 01 ∕20 03 ∕40 01 ∕4 ⎞ ⎟ ⎟ ⎟ ⎟ ⎠ . Decide whether each state is transient or recurrent. *18.5 Consider the Markov chain on states 1 to 4 with transition matrix ⎛ ⎜ ⎜ ⎜ ⎝ 1∕31 ∕31 ∕30 1∕31 ∕301 ∕3 1∕301 ∕31 ∕3 00 1 ∕21 ∕2 ⎞ ⎟ ⎟ ⎟ ⎠ . Decide whether each state is transient or recurrent. EXERCISES 301 *18.6 Consider the Markov chain on states 1 to 4 with transition matrix ⎛ ⎜ ⎜ ⎜ ⎝ 00 .80 .20 00 0 .30 .7 0.50 .50 0 0001 ⎞ ⎟ ⎟ ⎟ ⎠ . Decide whether each state is transient or recurrent. *18.7 For a two-state Markov Chain with transition matrix ( 1−pp q 1−q ) , find the limiting distribution in terms ofpandq *18.8 An indecisive diner enters a restaurant and is taken to a round table with five chairs. Unabletodecidewhichchairisbest,thedinerswitcheseveryminute,movingclock- wise with probabilityp and counterclockwise with probability 1−p. We can then consideraMarkovchain X n,thenumberofthechairoccupiedattime n,inminutes. Show that the condition ensuring a limiting distribution holds in this case. What is the limiting distribution? What can you say if there are four chairs rather than five? *18.9 A particle moves around a square with four vertices, numbered 0,1,2,3 going clockwise. It moves clockwise with probability 2∕3 and counterclockwise with probability 1∕3. Motion stops when the particle reaches vertex 3. Find the expected amountoftimethevertexwillspendatvertex2,startingfromeachofvertices,0,1,2. *18.10 ShowthatforeitheraperiodicorreducibleMarkovchain,givenanyinteger n,there exists some ordered pair (i,j) such that the entry in rowiand columnjofP n = 0. *18.11 Agolferhasaprobabilityof(1 −0.1n)ofmakingan n-footputt,where nislessthan 10. He adopts the following routine for practicing putts from three to seven feet. He begins with a three foot putt. Anytime he makes a putt, he tries one which is a foot longer and any time he misses, he goes back to a foot shorter putt. If he misses on the three-foot putt or makes the seven-foot putt, he repeats that distance. (a) Ifhecontinueshissessionforaverylongtime,whatisthemostlikelydistance that he will finish with? Estimate the probability of finishing at that distance. (b) Repeatpart(a)onlyassumingnowthatheispracticingputtsfromfourtoeight feet. 18.12 In a Poisson process, the probability that exactly one event will occur in any given hour is 3e −3. (a) What is the probability that exactly two events will occur in any 20-minute period? (b) Suppose you start observing the process at some point of time. What is the probability that it will be less than 10 minutes until an event occurs? 302 AN INTRODUCTION TO STOCHASTIC PROCESSES (c) Take a unit of time to be 1 hour. Starting at time 0, find the expectation and variance of the time of the fifth occurrence. 18.13 Suppose N(t) is a Poisson process such thatP[N(2) = 1] = 6e−6. (a) FindP[N(1) = 2]. (b) Identify the distributions (including parameters) ofWn, the associated waiting- time process, andTn, the time of thenth arrival. (c) FindP[(N(1) = 1 andN(3) = 3]. 18.14 Vehicles pass a certain marker on a highway in accordance with a Poisson process at the rate of 48 per hour; 25 % of the vehicles on the road are trucks and 75 % are cars. Suppose you observe this marker at a particular point of time. (a) Whatistheprobabilitythatexactlyfivevehicleswillpassinthefirst15minutes? (b) Giventhatexactlytwotruckspassedinthefirst5minutes,whatistheprobability that the fourth truck will pass in the first 10 minutes? (c) Given that exactly two trucks passed in the first 5 minutes, what is the expected value and the variance of the number of vehicles that passed in the first 10 minutes? 18.15 Thefollowinggivesinformationaboutthreecountingprocesses, N 1(t),N2(t),N3(t). Give reasons why each of these cannot be a Poisson process. (a) P[N 1(2)−N1(1) = 10] = 0.5,P[N1(3)−N1(2) = 10] = 0.4 (b) P[N2(1) = 1] = 0.5 P[N2(1) = 1 andN2(2) = 2] = 0.2 (c) P[N3(1) = 0] = 0.5, P[N3(2) = 1] = 0.3 18.16 Consider a Poisson process N(t) with rate 2 per time period. You are given that N(1.8) = 4. Using this information, answer each of the following. (a) What is the probability thatN(2.3) ≤ 6? (b) What is the probability that the fifth occurrence will be before time 2? (c) What is the expected time and the variance of the seventh occurrence? 18.17 At a subway station, eastbound trains and northbound trains arrive independently, both according to a Poisson process. On average, there is one eastbound train every 12 minutes and one northbound train every 8 minutes. Suppose you arrive at the subway station at a certain point of time and start observing trains. (a) What is the probability that exactly two eastbound trains will arrive in the first 24 minutes? (b) What is the probability that exactly two eastbound trains will arrive in the first 24minutesandexactlythreeeastboundtrainswillarriveinthefirst36minutes? EXERCISES 303 (c) Whatistheexpectedwaitingtime,inminutes,untilthefirsttrain(ofeithertype) arrives? (d) Whatistheprobabilitythatthefirsttraintoarriveiseastbound,andthenexttwo are northbound? (e) What is the probability that it will take at least 20 minutes for two northbound trains to arrive? 18.18 Events occur according to a Poisson process. Suppose that the expected number of occurrencesperhouris3,andlet Tn denotetimeinhoursofthe nthoccurrence.Find the expected value and variance ofT3 given that the first occurrence was at time 1 and the second occurrence was at time 2. 18.19 In a Poisson process, the probability is 0.60 that after an occurrence of the event, it willtakeatleast2monthsuntilthenextevent.Findtheprobabilitythatexactlyfour claims will occur within any 5-month period. 18.20 An insurer finds that out of a certain group of insured drivers, the accident rate over each 24-hour period rises from midnight to noon, and then declines until the following midnight. They decide that the number of accidents can be modelled by a nonhomogenous Poisson process where the intensity at timet is given by [1∕6−(12−t) 2∕1152], wheretis the number of hours since midnight. Find: (a) the expected number of daily accidents; (b) the probability that there will be exactly one accident between 6:00 a.m. and 6:00 p.m. 18.21 Suppose that X(t) is a Brownian motion with variance parameter 0.03. IfX(2) = 3, find the probability thatX(5) > 3.5. *18.22 Foracertainsecurity,each1unitinvestednowwillyieldtherandomamount e0.2B(t) at timetwhereB(t)isastandardBrowniannotion.Whatistheprobabilitythatsometime within the next five time periods, an initial investment will have doubled in value. *18.23 The price of a stock at timetis siven by S(t) = 15e𝜇t+0.3B(t), where B(t) is a standard Brownian motion. The quantity𝜇 is unknown. An investor buys the stock at time 3 and sells it for 20 a share at time 5. What is the probability that the investor lost money on this transaction? 19 Multi-state models 19.1 Introduction Multi-statemodelsareanattempttolookatavarietyoflifeinsuranceandannuitycontractsin aunifiedmanner,bymakinguseofMarkovprocesses.Asmotivation,takeanindividualnow age xand consider a two-state Markov chain, where the person is in state 0 (alive) or state 1 (deceased)atanytime.Lifeinsurancecontractsprovidebenefitsupontransferfromstate0to state 1, while life annuity contracts provide benefits as long as the process remains in state 0. Moregenerally,consideramultiple-decrementmodelfor( x)with mcausesoffailure.We canconsiderachainwith m+1states.State0meansthat(x )hasnotsuccumbedtoanycause and is often referred to as theactive state. Statej refers to having succumbed first to cause j. The insurance benefits discussed in Chapter 11 can be viewed as payments upon transfer from state 0 to other states. For still another example, consider a joint-life contract issued to (x) and (y). We can now take a chain with four states as illustrated in Figure 19.1. The arrows indicate that there are possible transitions from state 0 into state 1 or state 2, occasioned by the death of (y)o r(x) respectively, and then further transitions from state 1 or state 2 into state 3 when the second death occurs. The dotted line, showing a transition directly from state 0 to state 3, would not bepresentinouroriginalmodel,butwouldbethereifwewantedtoconsideracommonshock modelasdiscussedinSection17.6.Ajointlifeinsurancecanbeconsideredastwocontracts, one paying benefits upon transfer from state 0 to state 1, and the other paying benefits upon transferfromstate0tostate2.Ageneraltwo-lifeannuitycanbeconsideredasthreeseparate contracts, where theith contract,i= 0,1,2, pays benefits provided the process is in statei. This can be generalized to contracts involvingnlives where we will have 2 n states. Wecanofcourseimaginemoregeneralpatternsoftransition.Wemaywishtoinvestigate a more enriched multiple-decrement model where individuals can transfer between states several times. A disabled person might recover and re-enter the main group of lives. In our originalmodelweignoredwhathappenedtoalifeonceitleftthegroupforanycause,butwe FundamentalsofActuarialMathematics , Third Edition. S. David Promislow. © 2015 John Wiley & Sons, Ltd. Published 2015 by John Wiley & Sons, Ltd. Companion website: http://www.wiley.com/go/actuarial THE DISCRETE-TIME MODEL 305 Both alive State 0 (x) only State 1 (y) only State 2 Neither State 3 Figure 19.1 A two life multi-state model maywishtomodelthefactthatsomeoneleavingforacauseotherthandeathwillsubsequently die. There are many examples of insurance and annuity benefits applicable to this general case. When disability is one of the decrements, we may have a contract that pays benefits when a person becomes disabled, and then further benefits when a disabled person dies, and possibly additional payments that continue during disability. Indeed, a common provision in many life insurance policies is adisability premium waiverclause, which mean the person does not have to pay premiums during the time they are disabled. We view this as an annuity providing payments during a state of disability, which cease when there is a transfer back to an active state. These are only some of the possibilities, and we invite the reader to think of additional applications. In this chapter, we will investigate the general multi-state model. We first discuss the discrete-time model, where transitions between states can occur only at integer times. Fol- lowing that we deal with the more complicated continuous-time model, where we allow for transitions at arbitrary times. 19.2 The discrete-time model 19.2.1 Non-stationary Markov Chains The transitions in multi-state models are normally age-dependent. To handle this, we need to extend the concepts and notation that we introduced in Section 18.4.1 so that they apply to the non-stationary case. Foranon-stationaryMarkovChain,inplaceofthesingletransitionmatrix P,weneedfor each nonnegative integern, a matrixP n whoseijth entry ispij(n,n+1) The main quantities of interest are the probabilitiespij(k,n) of being in statejat timen conditional upon being in stateiat timek. In deriving (18.5), we do not need the fact that the matrices are the same, and the argument given is easily adapted to show that pij(k,k+n)i st h eijth entry ofPkPk+1 …Pk+n−1, (19.1) which reduces to (18.5) in the stationary case. Equation (18.6) now takes the form 𝜋k+n = 𝜋k PkPk+1 …Pk+n−1. 306 MULTI-STATE MODELS Unhealthy State 1 Deceased State 2 Healthy State 0 Figure 19.2 The healthy-unhealthy-deceased model For example, in our simple life-death chain the transition matrices are given by Pn = ( px+n qx+n 01 ) . In the multiple decrement example, the matrixPn will be similar to that above, except the first row will be (p(𝜏) x+n,q(1) x+n ,q(2) x+n ,…,q(m) x+n), and each other row will have 1 on the main diagonal and 0s elsewhere. We leave is as an exercise for the reader to write down the matrixPn in the joint two life case. We now introduce a multi-state model which is not covered by our previous cases of multiple lives or multiple-decrement theory, and to which we will refer to frequently in the rest of the chapter. It allows for atwo-way transitionbetween certain states, which is really wherethegeneralityofthemulti-statemodelisthemosthelpful.Thisisthechainasdepicted in Figure 19.2 where a life can be healthy or unhealthy, but we now allow for recovery from the unhealthy state. There are various applications, The unhealthy state could refer to being disabled, or sick, or a number of other possibilities for a state which we wish to distinguish somehow from the main group of lives, but which can transfer back to the main group. Example 19.1 Consider the model of Figure 19.2 and suppose that transition matrices for times 0, 1, 2 are given as follows; P 0 = ⎛ ⎜ ⎜ ⎝ 0.70 .20 .1 0.20 .60 .2 001 ⎞ ⎟ ⎟ ⎠ P 1 = ⎛ ⎜ ⎜ ⎝ 0.50 .30 .2 0.10 .60 .3 001 ⎞ ⎟ ⎟ ⎠ P 2 = ⎛ ⎜ ⎜ ⎝ 0.40 .30 .3 0.10 .50 .4 001 ⎞ ⎟ ⎟ ⎠ . What is the probability that a person who is healthy at time 0 will be unhealthy at time 2 and deceased at time 3? Solution. P 0P1 = ⎛ ⎜ ⎜ ⎝ 0.37 0.33 0.30 0.16 0.42 0.42 001 ⎞ ⎟ ⎟ ⎠ , THE DISCRETE-TIME MODEL 307 Using formula (19.1), the required probability equals p01(0,2)p12(2,3) = 0.33×0.4 = 0.132. 19.2.2 Discrete-time multi-state insurances Tomodelthegeneralcontractofthistype,wewilltakeaMarkovchainwithstatesnumbered from 0 toN. Suppose the process begins in stateaat time 0. Fix any two statesiand j, and consideracontractwhichpaysabenefitof b k attime k+1,providedatransferfromstate ito state joccurs between timekand timek+1. That is, the process was in stateiat timekand statejat timek+1. Notation If we have several different transfer benefits we will distinguish by writingbk as b(ij) k . In general, for quantities like benefits, premiums, reserves and expenses, we will let superscripts refer to states, and maintain our previous usage of subscripts to refer to time. This differs from our convention for probabilities where we use subscripts to refer to states, and write times in brackets. A convenient method of discussing multi-state benefits in the context of the stochastic model is to make use of indicator random variables. For each nonnegative integerk,l e tI k be therandomvariablethattakesthevaluesof0or1respectively,accordinglyasatransferfrom ito jhas not or has occurred between timekand k+1. The present value of the benefits on the contract is given by Z = ∞∑ k=1 bkv(k+1)Ik. (19.2) Since E(Ik) = pai(0,k)pij(k,k+1), (19.3) the actuarial present value is ∞∑ k=0 bkv(k+1)pai(0,k)pij(k,k+1). (19.4) Weagainhaveourfamiliarpatternwithasumofthree-termedfactors.Infact,wecanconsider each summand as a four-termed factor where the probability of receiving a payment is itself theproduct oftwofactors,theprobabilityoftransitiontostate ibytime kand theprobability of transition from stateito statejin the next period. Example 19.2 Consider the chain given in Example 19.1. A 3-year contract written on a healthy life provides for benefits at the end of the year of death. The death benefit is 3 if the person dies while healthy and 2 if the person dies while unhealthy. If interest is a constant 5%, find the actuarial present value. Solution. We can view this as two separate contracts, one paying 3 for transfer from state 0 to state 2, and the second paying 2 for transfer from state 1 to state 2. We use the matrices in 308 MULTI-STATE MODELS Example 19.1 and formula (19.1). For the first contract, p02(0,1) = 0.1, p00(0,1)p02(1,2) = 0.14, p00(0,2)p02(2,3) = 0.37×0.3 = 0.111, so that APV = 3[(0.1)(1.05)−1 +0.14(1.05)−2 +0.111(1.05)−3] = 0.9543. For the second contract, p01(0,1)p12(1,2) = 0.06, p01(0,2)p12(2,3) = 0.33×0.4 = 0.132 so that APV = 2[0.06)(1.05)−2 +0.132(1.05)−3] = 0.3369. Therefore, the total APV is 1.2912. Example 19.3 Take the same chain as in the previous example. Consider a contract that pays 1 at the end of any year of transfer from being healthy to being unhealthy, provided this occurs within the next 3 years. Find the expected value and variance of the benefits. Solution. We now have p 01(0,1) = 0.2, p00(0,1)p01(1,2) = 0.21 p00(0,2)p01(2,3) = 0.37×0.3 = 0.111, so that APV = 0.2(1.05)−1 +0.21(1.05)−2 +0.111(1.05)−3 = 0.4768. The variance calculation can not be done conveniently by adapting a version of formula (15.4). The situation here is different from the multiple decrement model where benefits are paidupontransitiontoanabsorbingstate.Inthiscase,therecanbebenefitspaidatmorethan onetime.Wemustinsteadusetheapproachtakeninthecurrentpaymentformulaforannuity variances. Note however that the covariance calculations will be more complicated than that givenby(15.11).Letting I k takethevalue1iftransferfromthehealthystatetotheunhealthy state occurs between timekandk+1, we have E(I0) = 0.2, E(I1) = 0.21, E(I2) = 0.111, E(I0I1) = E(I1I2) = 0, while E(I0I2) = p01(0,1)p10(1,2)p01(2,3) = 0.006, which is the probability of becoming unhealthy, recovering, and then becoming unhealthy again, thereby being paid at both time 1 and time 3. We can then calculate the covariances as Cov(I 0I1) =− 0.042, Cov(I 0I2) =− 0.0162 Cov(I 1I2) =− 0.02331, THE DISCRETE-TIME MODEL 309 leading to the calculation of the variance as (0.2)(0.8)(1.05)−2 +(0.21)(0.79)(1.05)−4 +(0.111)(0.889)(1.05)−6 −2[0.042(1.05)−3 +0.0162(1.05)−4 +0.02331(1.05)−5 = 0.2195. Suppose we wished to modify the contract of the previous example so that it paid off onlyonthe firstoccurrenceofbecomingunhealthy.Inoursimpleexample,thiscouldbedone readilyenoughbychangingtheprobabilityofabenefitpaymentattime3to0.105,subtracting 0.006,whichwehavecalculatedaboveastheprobabilityofthesecondoccurrence.Tohandle the problem in general, we need a more systematic approach. Consider a contract paying benefits upon the first transition from a statei 0 to a statej0. A method that works here ( as well for many other problems ) is to add additional states. The easiest way is to add a single new absorbing statez, and all transfers from statei0 to statej0 are redirected toz, and the contract is modified to pay upon a transfer fromi0 to z. To state this precisely, letp∗ denote probabilities in the augmented chain. To simplify the notation, fix a timenand writepij and p∗ ij for pij(n,n+1) andp∗ ij (n,n+1) respectively. Then p∗ is the same aspexcept that p∗ i 0j0 = 0, p∗ i 0z = pi0j0, p∗ zz = 1, p∗ zi = 0, i≠ z, p∗ iz = 0, i≠ z,i0 There is a more involved method which has the advantage that it preserves relationships betweenotherstatesthatmightbeneededinmorecomplexproblems.Thisistoadd N+1new statesi∗,i= 0,…,N,wherestate i∗ canbeconsideredasasortof‘clone’ofstate i.Transitions betweenun-starredstatesareasbeforeexceptthattransitionsfrom i0 toj0 aredirectedto j∗ 0 in placeof j0.Exceptforthat,therearenotransitionsbetweenthetwotypesofstates,starredand un-starred. Transitions between starred states are as given by the corresponding transitions betweentheirclones.Inthiscase, p ∗ i 0j0 = 0andforallorderedpairs(i, j)notequalto( i0,j0), p∗ ij = pij, p∗ i 0j∗ 0 = pi0j0, p∗ i ∗j∗ = pij, p∗ i ∗j = 0, p∗ ij ∗ = 0. (19.5) For example, if our original matrix for a two state model is given by P = ( a 1−a b 1−b ) , and we takei0 = 0,j0 = 1, the the matrix of the augmented chain, using the order 0,1,0∗,1∗ is P∗ = ⎛ ⎜ ⎜ ⎜ ⎝ a 00 1 −a b 1−b 00 00 a 1−a 00 b 1−b ⎞ ⎟ ⎟ ⎟ ⎠ . In addition to this, the provisions of the insurance are modified to pay upon a transfer from state i 0 to statej∗ 0 . After the first occurrence of such an event, there can never be a return to statei0, so subsequent occurrences are ruled out. 310 MULTI-STATE MODELS Remark Solving multi-state problems with large matrices can involve a great deal of cal- culation.Thereareanumberofcomputingpackagesthatcanassistwiththis.Oneconvenient example is the MMULT function of Excel ®. 19.2.3 Multi-state annuities We now look at annuity contracts associated with a chain. Suppose again that the individual begins in stateaat time 0. For a fixed statei, we can consider a contract that paysc k at time k provided the person is in statei at that time. LetIk now be the indicator random variable that takes the value of 1 or 0 according as the process is or is not in stateiat timek. Then the present value of the benefits is the random variable Y = ∞∑ k=0 ckv(k)Ik, from which we calculate immediately APV = ∞∑ k=0 ckv(k)pai(0,k). (19.6) To derive variances, we calculate for allm< n E(ImIn) = pai(0,m)pii(m,n). From this, we calculate the covariances and then proceed as in Example 19.3. Example 19.4 Consider again the chain given in Example 19.1. A contract on a healthy life provides for four yearly payments beginning at time 0. The amount of the payment is 1 if the person is healthy, or 2 if the person is unhealthy. If interest is a constant 5%, find the actuarial present value. Solution. We view this as two separate annuities. Contract 1 is for state 1 and contract 2 is for state 2. In addition to the matrixP 0P1 calculated in Example 19.1, we need P0P1P2 = ⎛ ⎜ ⎜ ⎝ 0.181 0.276 0.543 0.106 0.258 0.636 001 ⎞ ⎟ ⎟ ⎠ . The APV of contract 1 is 1+ 0.7 1.05 + 0.37 1.052 + 0.181 1.053 = 2.1586, and that of contract 2 is 2 ( 0.2 1.05 + 0.33 1.052 + 0.276 1.053 ) = 1.4564, giving a total APV of 3.6150. THE CONTINUOUS-TIME MODEL 311 Example 19.5 Suppose the contract in Example 19.2 is to be purchased by three level annual premiums of𝜋, which are payable only if the person is healthy. Calculate𝜋. Solution. Equating values of premiums and benefits, 𝜋(1+(1.05)−10.7+(1.05)−20.37) = 1.2912, which gives𝜋 = 0.6449. Thereareothertypesofannuitycontractsthatcanbehandledbythetrickofaddingstates. Suppose for example, a contract provides periodic annuity benefits starting when the process entersstate i 0 (orstartingattime0iftheprocessbeginsinstate i0),butwhichstopcompletely when the process leaves statei0, even if there is a subsequent return. The easiest approach here is to add an absorbing statezand all transitions from statei0 to other states are directed to statez, ensuring the process never returns toi0. If we want to preserve information about other transitions, the method is to add a clone for each state as we did in the insurance case. Transitions from statei 0 to any statej≠ i0 are redirected to the clonej∗, also ensuring that the process will never return to statei0. 19.3 The continuous-time model In a realistic situation, transitions between states need not occur at discrete intervals, but can happen at any point of time, and the benefits payable upon transition from one state to the another will be paid at the moment of transition. To model this situation, we need to consider continuous-time Markov processes. We still keep a finite number of states, but let time vary continuously. The definition parallels that which we make in Section 18.2, only we must consider all possible points of time. (We will follow the notational usage introduced in Section 18.5 of writing the time variable in brackets.) Definition19.1 Acontinuous-timeMarkovprocessisastochasticprocess X(t):0≤ t< ∞, whereeach X(t)isdiscrete,withthepropertythatgiven anysequence oftimes,0 ≤ t 0 < t1 < ⋯ < tn < tn+1, and any sequence (x0,x1,x2 …xn,xn+1) wherexi is a value ofX(ti)w eh a v e P(X(tn+1)=xn+1|X(tn)=xn,X(tn−1)=xn−1,…,X(t0)=x0)=P(X(tn+1)=xn+1|X(tn)=xn ) As before, for timess≤ tand any two statesi,jwe let pij(s,t) = P(X(t) = j|X(s) = i), which is the probability of reaching statejat timetwhen starting in stateiat times. 19.3.1 Forces of transition Withthecontinuoustimeframework,weareabletodefineanaloguesoftheforceofmortality inouroriginallife-deathmodel,andtheforcesofdecrementinthemultiple-decrementmodel. Aswehaveseeninearlierexamples,theprocessisoftenspecifiedbygivingtheseforces,and then the central problem is to use these to calculate the desired probabilities. 312 MULTI-STATE MODELS Definition 19.2 We define theforceoftransitionfromstateitostatejattimet as follows. Ifi≠ j 𝜇ij(t) = lim h→0+ pij(t,t+h) h , while 𝜇ii(t) = lim h→0+ pii(t,t+h))−1 h . Note that𝜇ii is always nonpostive, which may seem a bit strange at first, but it is easily explained if we consider, for example, the multiple-decrement case where state 0 refers to an active life. The negativity of𝜇 00 reflects the action of the other forces of decrement, which cause transferoutof state 0. An important fact is the following. Theorem 19.1 N∑ j=0 𝜇ij(t) = 0, foralliandt . Proof. Fix anyi and t. Refer to the littleo notation introduced in Section 18.3. From the definitions above, for all statesjand h> 0, pij(t,t+h) = h𝜇ij(t)+o(h),i≠ j, (19.7) and pii(t,t+h) = h𝜇ii(t)+o(h)+1. (19.8) Summing over alljgives 1 = h N∑ j=0 𝜇ij(t) = o(h)+1. Subtract 1 from each side, divide byh and take a limit ash approaches 0 to establish the conclusion. We now want to relate the forces of transition to the transition probabilities. This is much more complicated in our general setting than in the simple cases we looked at previously. Theorem 19.2 (Kolmogorov forward equations)Foranystatesi ,jandtimess < t 𝜕 𝜕tpij(s,t) = N∑ k=0 pik(s,t)𝜇kj(t) (19.9) Proof. Westartbyderivingthediscreteversionoftheaboveformula.Followingthereasoning used in Equation (18.4), we can deduce that for statesi,jtimes s< tand h> 0 pij(s,t+h) = N∑ k=0 pik(s,t)pkj(t,t+h), (19.10) THE CONTINUOUS-TIME MODEL 313 reflectingthefactthatinordertoreachstate jfromstate iinthetimeinterval s,t+hwemust first reach some statekby timetand then go from statekto statejin the nexthtime units. The system of equations given by (19.10) is known as theChapman-Kolmogorovequations. Substituting from (19.7) and (19.8) into the second factor of the summand, pij(s,t+h) = h N∑ k=0 pik(s,t)𝜇kj(t)+pij(s,t)+o(h). Subtractingpij(s,t) from both sides, dividing byhand taking a limit gives (19.9). We can give an intuitive explanation of (19.9) paralleling that given in the discrete case. View each side as a type of density function for reaching statejat timetwhen starting from state iat times. The right side reflects the fact that in order to accomplish this, we must first reach some statekat timetand then at that instant of time, transfer from statekto statej(or in the case thatk= j, not transfer back into some other state). Weviewthestatementofthetheoremasasystemofdifferentialequationsinthevariable t forafixedvalueof s.Theword‘forward’arisessinceourtransitionismovingforwardintime. There is a related system, the correspondingbackwardsequations, which will be developed in Exercise 19.6. There will normally be many solutions to a system of differential equations of this type. There will however often be a unique solution if we specify appropriate initial conditions. In our context, these conditions are the obvious requirements that for allt, p ij(t,t) = { 0, i≠ j 1, i= j. WehaveofcourseencounteredversionsoftheKolmogorovequationsbefore.Take s= 0. The simplest case withN = 1 gives Equation (8.15). In the multiple decrement model, the Kolmogorov equations produce Equation (11.10). These statements may not be completely obvious since the notation is somewhat different. It will be instructive for the reader to verify these claims. There is a nice matrix formulation of (19.9). Fixings,t we letP denote the transition matrixforthetimeinterval(s, t).Thisisthematrixwithanentryinthe ithrowand jthcolumn of p ij(s,t), as we had in the discrete case. We also have a corresponding matrixM in which the entry in theith row andjth column is𝜇ij(t). Mis often referred to as theintensitymatrix. We letP′ denote the matrix in which each entry is the partial derivative with respect tot of the corresponding entry inP. The Kolmogorov forward equations together with the initial conditions can then be expressed as P′ = PM, (19.11) and P= I fors= t. (19.12) Thefollowingisasimplebut usefulconsequence oftheforwardequations. Itgeneralizes a result from our life-death model, when we have a state, such as that of being alive, which cannot be entered from any other state. That is, for allj≠ iand s< t,w eh a v ep ji(s,t) = 0. We will refer to such a state as beinganti-absorbing. 314 MULTI-STATE MODELS In the following theorem, we adopt a formally weaker hypothesis involving forces rather than probabilities, which by the second statement turns out to be equivalent. Theorem 19.3 Letibeastatesuchthat 𝜇 ji(t) = 0forallj ≠ i,andallt.Then,fors < t, pii(s,t) = e∫ t s 𝜇ii(r)dr, and pji(s,t) = 0, forj ≠ i. Proof. From Theorem 19.2 we get 𝜕 𝜕tpji(s,t) = pji(s,t)𝜇ii(t), (19.13) afamiliardifferentialequationwhichwefirstencounteredin(8.15)andwhichiseasilysolved to give p ji(s,t) = K(s)e∫ t s 𝜇iird r forsomefunction K(s)independentof t,Since pii(s,s) = 1,and pji(s,s) = 0for j≠ i,wemust haveK(s) = 1i fj= ior 0 ifj≠ iand the conclusion follows. Another quantity of interest is the so-calledsojournprobability for a state. We define pii(s,t) = P[Xr = i,s≤ r≤ t|Xs = i]. Note that the sojourn probability will in general differ frompii(s,t), as the corresponding event requires the process to remain in stateifor an entire interval of time, rather than just at the endpoints. The two probabilities will however be the same for an anti-absorbing state. Since sojourn probabilities are featured prominently in the next section, we will attempt to find a way to evaluate them. Let qi(s,t) = pii(s,t)−pii(s,t) This is the probability that starting in stateiat times, the process is back in stateiat timet, having left stateisometime during the interval. Let vi(t) = lim h→0+ qi(t,t+h) h . Now we employ once again the technique of adding states. Given a process and a statei, we add a single new statei∗ which is a clone of statei. Transitions out of statei∗ follow the samepatternastransitionsoutofstate i,whiletransitionsthatoriginallycameintostate iwill be diverted to the cloned statei∗. Now to say that the new process is in stateiat a timesand also at a later timetmeans that it must have stayed there throughout as it cannot re-enter. In THE CONTINUOUS-TIME MODEL 315 addition, the forces of transition out of stateiremain the same in both processes. So we have that p∗ ii (s,t) = pii(s,t), 𝜇∗ ij = 𝜇ij, j≠ i,i∗ We do have to consider the quantity𝜇∗ ii∗.N o w p∗ ii ∗(s,t) = qi(s,t), since the new process will move from stateito statei∗ precisely when the original process moves from stateiback to stateihaving left stateiat some point. From Definition 19.2, 𝜇∗ ii∗(t) = 𝜈i(t). Applying Theorem 19.3 to stateiin the new process then leads to the formula pii(s,t) = e∫ t s (𝜇ii(r)−𝜈i(r))dr. This formula does not however allow one to calculate the term on the left exactly since𝜈i also involves sojourn probabilities. All we can say without further assumptions is that pii(s,t) ≤ e∫ t s 𝜇ii(r)dr. Suppose, however, that we have a process such that, for all statesiandt,h> 0 qi(t,t+h) = o(h). (19.14) Thisassumptionsaysthatitishighlyunlikelythatinasmalltimeintervalthereisamoveout of a state and back into it. It is the same idea that underlies a Poisson process as discussed in Section 18.3, where it was highly unlikely to have two occurrences of an event in a small time interval. When this assumption holds, we have thatv i(t) = 0 for allt and we can now state the following theorem. Theorem 19.4 Foraprocesssatisfying(19.14), p ii(s,t) = e∫ t s 𝜇ii(r)dr. Remark We can view the first statement in Theorem 19.3 as the special case of Theo- rem 19.4, whenqi is actually equal to 0. An alternative direct proof of Theorem 19.4 is outlined in Exercise 19.18. To summarize our conclusions in this section we have shown that given the forces of transition, we can in theory, solve a system of differential equations to arrive at the required probabilities that we are interested in for our applications. However, in practice, the solution canbedifficultorimpossibletoobtainexactly.Inthefollowingsubsectionswelookatvarious possibilities for obtaining or approximating the probabilities from the forces. 316 MULTI-STATE MODELS 19.3.2 Path-by-path analysis Assumethenthatwearegiven 𝜇ij(t)forall i,jandt,andwewanttodeterminetheprobability that starting in statei at any times, we will be in statej at timet. What we can do in a relatively straightforward manner is the following. Given any path going from stateito state j, we can derive a formula for the probability that starting in stateiat times, the process will be in statejat timet, having following the prescribed path exactly. Precisely, we are given a path 𝜋 which is a sequence of states,i0,i1,…,iM where i0 = iand iM = j. We then want the probability that the process will make a transition from stateito i1 followed by a transition from i1 to i2, followed by a transition fromi2 to i3 and so on, continuing in such a fashion, withoutvisitinganyotherstates,andfinallyreachingstate jatorbeforetime t,andremaining in statejuntil timet. Let us denote such a probability byp𝜋 i,j (s,t). These can be calculated by integration, using only the forces and sojourn probabilities. So assuming that (19.14) holds, we have reduced the problem to one of evaluating integrals. To illustrate, consider the simplest possible case of a path of length 1, that is𝜋 = (i,j). Then arguing much the same as we did in the single life and multiple decrement models we can write p 𝜋 i,j (s,t) = ∫ t s pii(s,r)𝜇i,j(r)pjj(r,t)dr. The above formula parallels formula (11.13) except that in that case all non-active states areabsorbing,soonecannotgetoutofthem,whichmeansthatthethirdfactorintheintegrand doesnotappearasitisequalto1.Torecap,theformulaaboveshowsthatinordertogofrom i tojalongtheone-steppath,threeeventshavetooccur.First,theprocessmustremaininstate iuntilsometime rbetweentime sandtime t.Thiseventhasprobability p ii(s,r).Itcannotfirst gotoanotherstateandreturn,sincethatwouldconstituteadifferentpathfromthegivenone. Secondly,theremustbeatransitiontostate jattime r.Wecanthinkoftheprobabilityofthis as being𝜇 i,j(r)dr. Finally, the process must remain in statejfrom timerto timet. It cannot leave statejand return as this would similarly constitute a different path. We get the required probability by integrating the product of these probabilities over all timesrbetweensandt. Asthepathlengthincreasestheformulagetsmorecomplicated.Consideratwo-steppath, 𝜋 = (i,k,j). Now we need a double integral. The formula is p𝜋 i,j (s,t) = ∫ t s ∫ t r1 pii(s,r1)𝜇ik(r1)pkk(r1,r2)𝜇kj(r2)pjj(r2,t)dr2 dr1. This looks extremely complex, but it simply records what are now five steps to meet the required conditions. The process must remain in statei until some timer1, then transfer to state k, then remain in statek until some timer2, then transfer to statej, and then finally remain in statejuntil timet. A similar calculation that arises for a given pathΠ, is to find the probability that starting in stateiat times, the process will transfer to statejbefore timethaving followed precisely the prescribed path. We get the same multiple integral as described above, except the final sojourn probability is omitted, since the process need only enter, but not necessarily remain in the final state on the path. Of course, when the final state is absorbing, the two problems are identical. THE CONTINUOUS-TIME MODEL 317 Example 19.6 Consider a heathy-unhealthy-deceased model as introduced in Figure 19.2. The forces of transition are constant with𝜇01 = 2, 𝜇02 = 1, 𝜇10 = 1, 𝜇12 = 3. Find the prob- ability that a healthy life will become unhealthy, and then die without recovering before time 1. Solution. We have𝜇 00 =− 3,𝜇11 =− 4.Thepathinquestionis 𝜋 = (0,1,2)andtherequired probability is p𝜋 02 (0,1) = ∫ 1 0 ∫ 1 r1 e−3r12e−4(r2−r1)3dr2dr1. The double integral evaluates to 6 4 [ 1−e−3 3 −e−4(e−1) ] = 0.4279. In general, a path of lengthn will require ann-dimensional integral with an integrand consisting of 2n+1 factors, so this can quickly become computationally infeasible. Aside fromthatthereisanotherdifficulty.Usually,whatwereallywantis pij(s,t).Insomecasesthis might be obtained by adding upp𝜋 i,j (s,t) for all possible paths𝜋 fromitoj. However, even in thesimplestofmodels,therecouldwellbeaninfinitenumberofsuchpathsandthisprocedure will not work. This will normally occur whenever there is a positive probability of two-way transitions, as in the healthy-unhealthy-deceased model where healthy people can recover. A person could go from a healthy state to the deceased state, by getting sick and dying, or getting sick, then recovering, then getting sick again and dying, or recovering twice before death, or any of a number of infinite possibilities. We need other approaches to calculating transition probabilities, which we discuss in the next two subsections. 19.3.3 Numerical approximation There are various ways to find a numerical approximation to the solution of the Kolmogorov equations. We will describe one approach that is fairly simple to implement. It is similar to Euler’s method, described in Section 8.9, for Thiele’s equation. For a fixed times, the goal is to approximate the probabilitiesp ij(s,t). Pick a small interval of timeh, and change our unit oftime,sothateachoriginaltimeunitisnow1 ∕hunits.(So,forexample,ifouroriginaltime unit was a year, andh= 1∕12, this changes the time unit to months.). We then essentially define a discrete time multi-state model, which will in generally be nonstationary. We will define one step transition probabilities for this model by simply looking at formulas (19.7) and(19.8),whichformedthebasisoftheKolmogorovequations,onlyomittingthe o(h)terms in each case. Since these terms becomes very small ash does, we can hope that for small enough h, we obtain a good approximation. We also switch the origin of time so that the original times is now time 0 and the original times+mh for an integerm is now timem. Wethentakeourapproximatingdiscretenon-stationaryMarkovchaintobethatinwhichthe transition probabilities, denoted bỹp,a r eg i v e nb y ̃p ij(m,m+1) = h𝜇ij(s+mh),i≠ j, ̃pii(m) = h𝜇ii(s+mh)+1. 318 MULTI-STATE MODELS We then obtain transition matrices (̃Pm) which have for an (i,j)th entry, the quantity ̃pij(m,m+1.) Anotherwayoflookingatthisisthatwesimplygetthematrix( ̃Pm)bytakingtheintensity matrixM(s+mh),multiplyingallnon-diagonalentriesby handadjustingthediagonalentries so that the rows add to 1. When t= s+mhfor some integermwe can then obtain approximations by pij(s,t) ≐ the (i,j)th entry of̃P0 ̃P1 ⋯ ̃Pm−1. Example 19.7 In a chain with three states, you are given that 𝜇01(t) = 𝜇10(t) = 0.09(1+t), 𝜇02(t) = 0.18(1+t), 𝜇12(t) = 0.09, 𝜇20(t) = 𝜇21(t) = 0. Find an approximation top01(1,2) using the method described above withh= 1∕3. Solution. We have M(1) = ⎛ ⎜ ⎜ ⎝ 0.46 0.18 0.36 0.18 073 0.09 001 ⎞ ⎟ ⎟ ⎠ . so that ̃P 0 = ⎛ ⎜ ⎜ ⎝ 0.82 0.06 0.12 0.06 0.91 0.03 001 ⎞ ⎟ ⎟ ⎠ and similarly ̃P 1 = ⎛ ⎜ ⎜ ⎝ 0.79 0.07 0.14 0.07 0.90 0.03 001 ⎞ ⎟ ⎟ ⎠ , ̃P 2 = ⎛ ⎜ ⎜ ⎝ 0.76 0.08 0.16 0.08 0.89 0.03 001 ⎞ ⎟ ⎟ ⎠ . Then the approximation top 01(1,2) is the entry in row 0 and column 1 of̃P0 ̃P1 ̃P2, which is 0.15131. 19.3.4 Stationary continuous time processes One case where we can make some progress in solving the Kolmogorov equations directly is thatofa stationaryprocessasdefinedinChapter18.Wethenhavethateach 𝜇 ij(t)isaconstant, denoted by𝜇ij. Moreover, the process will be determined by the quantitiespij(t) = pij(0,t) since the stationary condition implies thatpij(s,t) = pij(t−s). As an example, we can write down completely the solution for the general two-state stationary process. Suppose the intensity matrix is M = ( −𝜇𝜇 𝜈 −𝜈 ) . THE CONTINUOUS-TIME MODEL 319 Let𝜌(t) = e−t(𝜇+𝜈).Thenthetransitionmatrix P(t),whichhastheentryof pij(t)inthe ithrow andjth column, is given by P(t) = (𝜇 +𝜈)−1 ( 𝜇𝜌(t)+𝜈 −𝜇𝜌(t)+𝜇 −𝜈𝜌(t)+𝜈𝜈 𝜌 (t)+𝜇 ) (19.15) We can verify thatPsatisfies equations (19.11) and (19.12) by direct calculation, noting that d𝜌(t)∕dt) =− (𝜇 +𝜈)𝜌(t), and that𝜌(0) = 1. Forthegeneralstationaryprocess,solutionsto P′ = PMcanbegeneratedfrom eigenvec- torsof M. These are non-zero vectorsa = (a0,a1,…,aN)f o rw h i c h aM = 𝜆a for some constant𝜆 known as aneigenvalue of M. (The left hand side above is a matrix multiplicationinwhichweview aasa1 ×N+1matrix.Accordingtotheconventionwehave usedforthetransitionprobabilities,thevectorappearsontheleft.)Toobtaineigenvectors,we first find the eigenvalues as those constants for which the determinant of (𝜆I−M) = 0. This involvessolvingapolynomialequationin 𝜆.Wecanthensolvefortheresultingeigenvectors. Examples will follow. Note now that given any such eigenvectora and any fixedi, the matrixP that has all zero entries except for anith row of (a 0e𝜆t,a1e𝜆t…aNe𝜆t) is a solution toP′ = PM.T h i s follows since theijth entry ofPM will be𝜆aje𝜆t which is the derivative of the corresponding entry inP. Note next that given any finite number of solutions toP′ = PM, the linearity implies that any linear combination of these solutions will also be a solution. We therefore seek a linear combination that will satisfy the initial conditions. This can always be done in the case that we can findN linearly independent eigenvectors. We illustrate with a simple example. Example19.8 ForthedataasgiveninExample19.6,findtheprobabilitiesofthefollowing events. (a) A life now disabled will be active at time 0.5. (b) A life now active will be deceased at time 1. Solution. The intensity matrix is M = ⎛ ⎜ ⎜ ⎝ −32 1 1 −43 00 0 ⎞ ⎟ ⎟ ⎠ . We can see immediately that 0 is an eigenvalue with an eigenvector of (0,0,1). We can also note that since the rows ofP add up to 1, the rows ofM add to 0, and the last row ofP is (0,0,1), it is sufficient to solve the reduced systemP r′ = PrMr in which the superscript rindicates a matrix with the rowN and columnN removed. So we can consider eigenvectors of Mr = ( −32 1 −4 ) . 320 MULTI-STATE MODELS The determinant of𝜆I−Mr is 𝜆2 +7𝜆+10 giving eigenvalues of−5 and−2. Solving, we find respective eigenvectors of (1,−2) and (1,1). (Note that eigenvectors are not unique as they can be multiplied by any non-zero constant.). To satisfy the initial conditions, we want the first row ofPr to be the particular linear combination of the vectorsa = e−5t(1,−2) andb = e−2t(1,1) which gives the vector (1, 0) when t= 0. We solve this to get a first row vector of (1∕3)a+(2∕3)b. For the second row, we want a linear combination that gives the vector (0,1) whent= 0. We solve this to get a second row vector equal to (−1∕3)a+(1∕3)b. Transition probabilities are then given by p00(0,t) = 1 3e−5t + 2 3e−2t, p01(0,t) =− 2 3e−5t + 2 3e−2t, p10(0,t) =− 1 3e−5t + 1 3e−2t, p11(0,t) = 2 3e−5t + 1 3e−2t. The answers to the particular questions asked are (a) p10(0,0.5) = 0.0953. (b) 1−p00(0,1) −p01(0,1) = 0.8218. (The answer to part (b) is of course larger than that of Example 19.6 since here we allow for an arbitrary number of recoveries from being sick, as well as death from a healthy state. 19.3.5 Some methods for non-stationary processes The above procedure can also be used solve the problem of determining probabilities from the forces, when these are ‘piecewise constant’. We can find transition matrices for each time interval on which the forces are constant, and then use the discrete time technique. The followingsimpleexampleillustratestheprocedure.Itinvolvesonlytwotimeintervalsbutthe method can easily be extended. Example 19.9 For a two-state model, we have forces of transition given by 𝜇 01(0,t) = { 1, if 0 ≤ t< 2, 2i f 2 ≤ t< 3. 𝜇10(0,t) = { 2, if 0 ≤ t< 2, 3i f 2 ≤ t< 3. Find the probability that the process will be in state 1 at time 2.5 given that it is is state 0 at time 0. Solution. p 01(0,2.5) = p00(0,2)p01(2,2.5) +p01(0,2)p11(2,2.5). THE CONTINUOUS-TIME MODEL 321 From the matrix (19.15), we have p00(0,2) = 2+e−6 3 , p01(0,2) = 1−e−6 3 p01(2,2.5) = 2−2e−2.5 5 , p11(2,2.5) = 2+3e−2.5 5 Substituting, the required probability is 0.39446. One method for handling a perfectly general chain is to approximate it by a piecewise continuousone,andthenusethemethodoutlinedabove.Todoso,onecanchooseasmalltime unit, and approximate the forces of transition by replacing them with those that are constant on each time interval, possibly using the midpoint value. 19.3.6 Extension of the common shock model Asmentionedintheintroduction,multiplelifetheory,multipledecrementtheory,orthemore general model discussed in Chapter 17 can all be looked at in a multi-state framework. In the standard cases where failure times are independent, this does not normally provide any advantageovertheconventionaltreatmentsthatwehavedescribedpreviously.Themulti-state approach,however,canbeusefulinmodellingcertaintypesofdependence.Asanillustration, we will revisit the common shock model of Section 17.5 and discuss some possible refine- ments.Weconfinetheanalysistothecaseoftwotypesoffailure,thefirstwithfailuretimeequal tomin(T ∗ 1,Z)andthesecondwithfailuretimeequalto min(T∗ 2,Z).Failureforthetwotypesis dependentduetothecommonshockbuttherewillbeadditionalsourcesofdependencewhen T ∗ 1 andT∗ 2 arethemselvesnotindependent.Anexampleofthisisthe‘broken-heart’syndrome mentioned in Section 10.2 where one type of failure can hasten failure of the other type. We consider a multi-state model with four states, as given in Figure 19.1, but adapted to cover general failure times. State 0 means that neither type of failure has occurred, state 1 meansthatonlythesecondcauseoffailurehasoccurred,state2meansthatonlythefirsttype offailurehasoccurred,andstate3meansthatbothtypesoffailurehaveoccurred.(Thestates 1 and 2 then are labelled by the number of the surviving type.) Let𝜇 i(t) denote the hazard function forT∗ i , and𝜌(t) denote the hazard function forZ. Suppose that forces of failure are of the form. 𝜇01(t) = 𝜇2(t), 𝜇02(t) = 𝜇1(t), 𝜇03(t) = 𝜌(t), 𝜇13(t) = 𝜇1(t)+𝜌(t)+𝜖1(t), 𝜇23(t) = 𝜇2(t)+𝜌(t)+𝜖2(t) for some functions𝜖1(t) and𝜖2(t). When these are 0, we have precisely the common shock model. In this more general setting, we use𝜖i(t) to build in some dependence betweenT∗ 1 andT∗ 2 by letting the distributions change after the first failure. The ‘broken-heart syndrome’ wouldcallforpositivevaluesof 𝜖i(t).Ontheotherhand,theremaybecaseswhereweassume that the prospects for failure typeiimproves after the first failure, and we would reflect this w i t han e g a t i v e𝜖i(t). Probabilities in this model can all be calculated by the procedure of Section 19.3.2 since thereareatmostthreepathsbetweenanytwostates.Moreover,sincenostatecanbere-entered once the process leaves it, we can use Theorem 19.4 for the sojourn probabilities. 322 MULTI-STATE MODELS For the following examples we assume that all forces areconstant. Example 19.10 Calculate the probability that at timen, the first type of failure will have occurred but not the second. Solution. From Theorem 19.1, 𝜇 00 =− (𝜇1 +𝜇2 +𝜌), 𝜇22 =− (𝜇2 +𝜌+𝜖2). We want p02(0,n) = ∫ n 0 p00(s)𝜇02(s)p22(s,n)ds= 𝜇1 ∫ n 0 e−(𝜇1+𝜇2+𝜌)se−(𝜇2+𝜌+𝜖2)(n−s) ds. The integral is easily evaluated to give p02(0,n) = 𝜇1 [ e−(𝜇2+𝜌+𝜖2)n −e−(𝜇1+𝜇2+𝜌)n 𝜇1 −𝜖2 ] . (19.16) Asacheck,supposethat 𝜖1 = 𝜖2 = 𝜌 = 0,sowesimplyhavetwoindependentexponential failure times. The answer reduces to e−n𝜇2(1−e−n𝜇1), which is just the probability that the first type of failure occurred before timenmultiplied by the probability that the second type did not occur before timen. Example 19.11 Find the probability that the second cause of failure occurs before timen and after the first cause of failure. Solution. IfΠis the path (0,2,3) we want, p Π 03 (0,n) = ∫ n 0 p02(0,t)(𝜇2 +𝜌+𝜖2)dt, (aspecialcaseofformula(19.17)inthenextsection.)Substitutingfrom(19.16)andintegrat- ing, the answer is 𝜇 1(𝜇2 +𝜌+𝜖2) 𝜇1 −𝜖2 [ 1−e−n(𝜇2+𝜌+𝜖2) 𝜇2 +𝜌+𝜖2 − 1−e−n(𝜇1+𝜇2+𝜌) 𝜇1 +𝜇2 +𝜌 ] When 𝜖2 = 0 we recover the answer of Example 17.12 (a). 19.3.7 Insurance and annuity applications in continuous time Consider an insurance contract that provides for two fixed statesj and k, payments at the moment of transfer whenever a transfer occurs from statejto statek. The amount paid for a THE CONTINUOUS-TIME MODEL 323 transferattime twillbe bt (denotedby b(jk) t ifwewishtodistinguishbetweenseveraltransfer benefits).Supposetheprocessisinstate iattime0.Thentheactuarialpresentvalueparallels the result we saw in the multiple decrement case and is given by ∫ ∞ 0 btv(t)pij(0,t)𝜇jk(t)dt. If benefits are to be paid on only the first transfer from statejto statek, then this can be handled by adding new states, exactly as we did in the discrete case. Similarly, for a contract that provides payments made continuously at the periodic rate of ct (writtenas c(j) t ifneededtodistinguishstates)attime t,providedtheprocessisinstate jthe present value is given ∫ ∞ 0 ctv(t)pij(0,t)dt. Some annuities may make use of the sojourn probabilities. Suppose, for example, that at time 0, the process is in statei, and payments at the periodic rate ofct are paid as long as the processremainsinstate i.Allpaymentsstopuponthefirstexitfromstate i.Thepresentvalue is given by ∫ ∞ 0 ctv(t)pii(0,t)dt. It is possible to fit the continuous time insurances and annuities into a stochastic model, and calculate variances aswell as expected values, aswe didinthediscretecase. Wewillnot pursuethishere.Ingeneral,thiswillrequireintegrationoffunctionswhosevaluesarerandom variables rather than definite numbers. This presents both theoretical and computational difficulties. More general probabilities can be deduced from the insurance formulas by the standard method we described in Section 10.8.4 of taking zero interest and benefit functions that take the value 1 or 0. For example suppose the process is in statei. The probability that withinn years it will at some point make a transfer from statejto statekis given by ∫ n 0 pij(0,t)𝜇jk(t)dt. (19.17) Example 19.12 An insurance contract based on the model in Example 19.8 provides for a benefit at timet of e.04t provided that a person now healthy dies while unhealthy. The force of interest is a constant 5%. Find the actuarial present value. Solution. This is given directly by ∫ ∞ 0 e0.04te−0.05tp01(0,t)𝜇12(t)dt= 3∫ ∞ 0 e−.01t ( 2e−2t −2e−5t 3 ) dt= 0.596. The following is an example which can be thought of as a generalization of contingent insurances. Suppose we have a path of statesΠ=i 0i1 …iM as described in Section 19.3.2. At time 0, the process is in statei0 and an insurance contract provides a payment of 1 at the momentthattheprocessentersstate iM,havingpreviouslyfollowedpreciselythepath Π.F or 324 MULTI-STATE MODELS example, consider the chain covering the two lives (x) and (y) as given in the introduction and the pathΠ= {0,1,3}. The contract based on this path is the second death-contingent insurance paying upon the death of (x) if this occurs after the death of (y). Example 19.13 Suppose that the force of interest and all forces of transition are constant. Find a formula for̄AΠ, the actuarial present value of the insurance based on the pathΠ. Solution. Tosimplifynotation,let vj = 𝜇ijij+1 and𝜇j =− 𝜇ijij Then,followingtheexplanation for the calculation of probabilities in section 19.3.2, we can set up the following multiple integral in whichr j denotes the time of transfer from stateij−1 to stateij. ̄AΠ = ∫ ∞ 0 ∫ ∞ r1 …∫ ∞ rn v0v1 …vM−1e𝜇0r1e𝜇1(r2−r1) ⋯e𝜇M(rN−rM−1)e−𝛿rMdrNdrM−1 ⋯dr1. The integration can be carried out in a straightforward manner to give ̄AΠ = v0v1 …vM (𝜇0 +𝛿)(𝜇1 +𝛿)⋯(𝜇M−1 +𝛿). Theformulaisquiteeasytorememberandtocomputefrom.Thenumeratoristheproduct of all the forces of transition along the path. The denominator is a product of factors, one for eachstateonthepath,exceptforthefinalone.Eachsuchfactoristhesumofalloutgoingforces fromthatstateplus 𝛿.Wecantake 𝛿 = 0togivetheprobabilitythatthepath Πwillbefollowed. NotethatFormula(8.28),Example10.5andFormula(10.25)withsubstitutionfromthese first two are special cases of the above. 19.4 Recursion and differential equations for multi-state reserves The basic idea of calculating reserves for multi-state contracts is the same as we have seen, exceptthatthereservemustbecalculatedforeachstate,(anideawehavealreadyencountered in Section 10.4.2). We let kV(j) denote the reserve at timek when in statej. Given that the process is in a certain state, the reserve at any time is as usual, the present value of future benefits less the present value of future premiums. Recursion and differential equation formulas can become more complicated. We start first by looking at the discrete time case, where we assume transfer benefits are payable at the end of the period of transfer. Example19.14 Considerthehealthy-unhealthy-deceasedmodel,andsupposethatwehave a stationary chain with transition matrix P = ⎛ ⎜ ⎜ ⎝ 0.60 .20 .2 0.20 .60 .2 001 ⎞ ⎟ ⎟ ⎠ . RECURSION AND DIFFERENTIAL EQUATIONS FOR MULTI-STATE RESERVES 325 An insurance contract provides for payments of 2 when a healthy life becomes disabled, 3 when a healthy life dies, and 4 when a disabled life dies. All benefits are paid at the end of the year of the change of state. This is paid for by a single premium. Assume the interest rate=0. Find the reserves at timek,f o rk> 0. Solution. Thiscanbedonebyrecursion,butwewillfirstdoitinamorecomplicatedway,so thattheeaseoftherecursionformulamethodcanbebetterappreciated.Wecouldconsiderthis as three separate contracts, but it is just as easy to do it all at once. In view of the stationarity, the reserve at timekfork> 0 will be independent of the time. It will however depend on the state at timek.L e ta= kV(0) and b= kV(1). In view of the single premium,awill just equal the present value of future benefits given the state is 0. Therefore, summing over all possible transfer timesj,w eh a v e a= 2 ∞∑ j=0 p00(0,j)p01 +3 ∞∑ j=0 p00(0,j)p02 +4 ∞∑ j=0 p01(0,j)p12. (19.18) Now, in this case, the matrixP is of a particular simple form. Namely P = ⎛ ⎜ ⎜ ⎜ ⎝ c+d 2 c−d 2 1−c c−d 2 c+d 2 1−c 00 1 ⎞ ⎟ ⎟ ⎟ ⎠ . It is not hard to see that P k = ⎛ ⎜ ⎜ ⎜ ⎜ ⎝ ck+dk 2 ck−dk 2 1−ck ck−dk 2 ck+dk 2 1−ck 001 ⎞ ⎟ ⎟ ⎟ ⎟ ⎠ , Inthepresentcase,wehave c= 0.8,d= 0.4.Substitutingin(19.18),weeasilysumtheinfinite geometric progressions to get a= 21 2 ( 1 0.2 + 1 0.6 ) 0.2+21 2 ( 1 0.2 + 1 0.6 ) 0.2+41 2 ( 1 0.2 − 1 0.6 ) 0.2 = 14 3 , We will leave it to the reader to verify similarly thatb= 13∕3. Wenowlookatgeneralrecursionformulas.Supposewehaveacontractprovidingforeach i,j,benefitsof b(ij) k attime k+1providingthereisatransferfromstate itostate jbetweentime kand timek+1. If we want to include expenses of payment, these can be incorporated into the transfer benefits. We can assume that for each statei,w eh a v eb(ii) k = 0, since payments for remaining in a state will be handled by annuity type benefits. We let𝜋(i) k denote thenet paymentcollectedattime kassumingtheprocessisinstate i.Thesewillbethepremiumless 326 MULTI-STATE MODELS any annuity payments paid out and less any expenses if applicable. Then if the system is in stateiat timek, k+1V(i) = ( kV(i) +𝜋(i) k ) − ∑ j≠i (pij(k,k+1) ( b(ij) k +k+1V(j) −k+1V(i) ) (19.19) Notethatthisissomewhatmorecomplicatedthanpreviousrecursionformulas,wherethe transfer was always to the deceased state, for which no reserves are required. In this more general case, upon transfer from stateito statej, we pay out the net amount at risk, and then also must set up the reserve required for statej, necessitating the extra term of k+1V(j) above. The following rearranged version of the recursion formula is instructive as well as being well suited to computation. For each statei, ( kV(i) +𝜋(i) k ) (1+ik) = ∑ j pij(k,k+1) ( b(ij) k +k+1V(j) ) . This simply says that the amount accumulated at the end of a period must provide for all the benefits paid out at that time, as well as all the reserves that are needed in the various states. Solving this general recursion to get reserves is not as straightforward as the cases we have previously looked at where reserves were all 0 except for one state. In general, we need to solve a system of linear equations rather than a single equation. Example 19.15 Solve the previous example by recursion. Solution. Substitute in (19.19) to get a= a−0.2(2+b−a)−0.2(3−a), b= b−0.2(a−b)−0.2(4−b), (19.20) which simplifies to 0.4a−.0.2b= 1, −0.2a+0.4b= 0.8, and this is solved to give a= 14∕3, b= 13∕3. Consideravariationontheabovewherealevelpremiumof 𝜋 ispaidaslongastheperson ishealthy.Wenowneeda 𝜋 ontherighthandsideofthefirstequationin(19.20)andsolving will lead to a= 14−10𝜋 3 b= 13−5𝜋 3 . Interestinglyenough,wecannowcalculatetheequivalenceprinciplepremium.Inviewof the stationarity, this must be the value of𝜋 which makesa= 0. This will be𝜋 = 7∕5 and the corresponding reserve for the unhealthy lives will be 2. The value of 7∕5 can also be verified PROFIT TESTING IN MULTI-STATE MODELS 327 directly by dividing the benefit present value of 14∕3 by the value of an annuity of 1 per year while the life is healthy. This is calculated as in Example 19.19 by summing the appropriate geometricprogressiontogive(1 ∕2)(1∕.2+1∕.6) = 10∕3.Thedisabledstatereserveof2can always be verified since 2=0.6(2) + 0.2(4), showing that 2 will provide for the same reserve for those who remain disabled and a death benefit of 4 for those who die. Inthecontinuouscase,wehaveThiele’sdifferentialequationsforthemulti-statereserves. Consider a contract providing benefits, including accompanying expense, ofb (ij) t at timet for transfer from statei to statej. Net payments are are collected continuously, where the periodic rate of payment at timetis 𝜋(i) t when the system is in statei. We then have a system of differential equations, one for each statei, as follows. d dttV(i) = 𝛿tt V(i) +𝜋(i) t − ∑ j≠i 𝜇ij(t) ( b(ij) t +tV(j) −tV(i) ) which we can also write as d dttV(i) = 𝛿tt V(i) +𝜋(i) t − ∑ j 𝜇ij(t) ( b(ij) t +tV(j) ) , (19.21) by invoking Theorem 19.1. 19.5 Profit testing in multi-state models WenowrevisitSection12.4.3andadaptittothemulti-statecase.Aswedidinthatsection,we will include notation for all the expenses, rather than incorporating them into other symbols. Suppose we have a general insurance contract based on a discrete time multi-state model providing for benefits ofb (ij) k at timek+1 for transfer from statei to statej between time k and k+1 and annuity benefits ofc(i) k paid at timek when in statei. We have a profit test basis at timekconsisting of premiums𝜋(i) k , percentage of premium expenses ofr(i) k , periodic expenses ofe(i) k , all paid at timek when in statei, and in addition a transition matrixPk, interest ratesik, and transfer expenses ofe(ij) k paid at timek+1 for a transfer from stateito state jbetween timekand timek+1. The profit testing procedure will conform precisely to the principles outlined in the single-life case. The profit for the period running from timek tok+1, for a policy in stateiat timekis given by Pr(i) k+1 = ( kV(i) +𝜋(i) k −c(i) k −e(i) k ) (1+ik)− ∑ j (pij(k,k+1) [ (b(ij) k +e(ij) k +k+1V(j) ] Note that this is just a straightforward generalization of the formula (12.4b) with the modification that we must provide reserves for the states transferred into, as noted in the Section 19.4. Example 19.16 An insurance policy issued to a healthy life provides for death benefits at the end of the year of death of 100000 if death occurs while healthy or 70000 if death occurs while unhealthy. In addition there are yearly payments of 30000 provided the insured 328 MULTI-STATE MODELS is unhealthy. You are given reserves as follows, with state 0 being healthy and state 1 being unhealthy. 5V(0) = 25000, 5V(1) = 75000, 6V(0) = 30000 6V(1) = 60000 The profit test basis has annual premium of 15,000 paid while healthy, an interest rate of 10%, periodic expenses paid each year of 200 if healthy or 400 if unhealthy, a transition matrix for the year running from time 5 to time 6 as the matrixP 0 of Example 19.1, and expenses for paying a death claim of 1% of the face amount. Find the profit Pr6 both when the policyholder is healthy at the end of 5 years and when the policy holder is unhealthy at the end of 5 years. Solution. Pr (0) 6 = (25000 +15000 −200)(1.1)−0.7(30000) −0.2(60000) −0.1(100000(1.01)=680. Pr(1) 6 = (75000 −30000 −400)(1.1)−0.2(30000) −0.6(60000) −0.2(70000(1.01) =−7080 19.6 Semi-Markov models In this section, we discuss briefly the problem that arises when the Markov condition does not hold. This is a frequent occurrence, since this will happen whenever the probabilities of movement from one state to another depend on the length of time elapsed since entering the current state. These are known assemi-Markov models. A typical example is found in the healthy-unhealthy-deceasedmodel.Thelikelihoodofeitherrecoveryordeathforaunhealthy person will certainly depend on how long they have been in the unhealthy state, a fact which was not considered in our previous models. We will not discuss these in detail. One method of approach is to approximate the semi- MarkovmodelbyaMarkovchain,bydividingstatesupintoseveralsub-states.Forexample, in place of a single statehfor being unhealthy we could have statesh 1,h2,…,hk where the only transitions with positive probability between this collection of states is fromhi to state hi+1. Therefore, a higher subscript is indicative of a longer period of being unhealthy. The probabilitiesoftransitiontobothrecoveryanddeathcouldthendifferbetweenthesub-states. Notes and references For a more detailed account of multi-state theory than we have presented here, see Norberg (2008). Transition probabilities, which involve four variables, can be written in a myriad of ways, and various choices of notation are found in different works. In much of the actuarial literature, the standard form of notation is maintained. The basic symbol used is tpij x to refer to the probability that a life now agexand in stateiwill be in statejat timet. In the notation ofthischapter,wewouldconsidertheage(x )fixedandwritetheaboveprobabilityas pij(0,t), following the typical usage found in much of the probability theory literature. The actuarial notation has the advantage of following classical actuarial usage for probabilities, but seems EXERCISES 329 restrictedtomodelswhichrefertoabodyoflives,andnotdirectlyapplicabletomoregeneral situations. See Jones (1997) for an interesting application of multi-state models. Exercises 19.1 RedoExample19.2,butassumenowthatinthefirstyear,theprobabilitythatahealthy person will become ill is 0.3 rather than 0.2, and the probability that they will remain healthy is 0.6 rather than 0.7. 19.2 Consider a Markov chain with transition matrices for the first three time periods given by P 0 = ⎛ ⎜ ⎜ ⎝ 1∕31 ∕31 ∕3 1∕31 ∕31 ∕3 1∕31 ∕31 ∕3 ⎞ ⎟ ⎟ ⎠ , P 1 = ⎛ ⎜ ⎜ ⎝ 1∕21 ∕20 01 ∕21 ∕2 1∕21 ∕20 ⎞ ⎟ ⎟ ⎠ , P 2 = ⎛ ⎜ ⎜ ⎝ 1∕21 ∕41 ∕4 1∕21 ∕41 ∕4 1∕21 ∕41 ∕4 ⎞ ⎟ ⎟ ⎠ . Interest rates are given byi 0 = 0.05, i1 = 0.06, i2 = 0.07. Find the APV of each of the following contracts. (a) Attime0,theprocessisinstate0.Acontractprovidesforpaymentsattheendof the period of a transfer from state 0 to state 1 if this occurs within three periods. The payment is 100 in the first year, 200 in the second year, and 300 in the third year. Note that more than one payment can be made. (b) At time 0 the process is in state 0. A contract provides for a payment of 1000 at timek,k= 0,1,2,3, provided the process is in state 0 at timek. 19.3 A transition matrix for a three-state homogeneous Markov chain is given by P = ⎛ ⎜ ⎜ ⎝ 0.40 .30 .3 0.70 .20 .1 0.20 .70 .1 ⎞ ⎟ ⎟ ⎠ Aprocessstartsinstate0.Annuitycontract1providesforperiodicpayments,provided the process is in state 1. Annuity contract 2 provides for periodic payments which starts at the end of the period in which process enters state 2, but stop completely upon exit from state 2 and do not begin again, even upon subsequent return. For each contract, find the probability that a payment is made at time 3. 19.4 A transition matrix for a four-state homogeneous Markov chain is given by P = ⎛ ⎜ ⎜ ⎜ ⎝ 0.10 .20 .30 .4 0.30 .10 .30 .3 0.50 .20 .10 .2 0.40 .30 .20 .1 ⎞ ⎟ ⎟ ⎟ ⎠ . 330 MULTI-STATE MODELS Attime0,theprocessisinstate0.Acontract provides forpayments attheendofthe year of thefirsttransition to state 3 from any other state, provide this occurs within 4 years. The amount of the payment is 100 if the transition is from state 0, 200 if the transition is from state 1, or 300 if the transition is from state 2. Level net annual premiums are paid beginning at time 0, and continuing until the time of transition to state 3. The interest rate is a constant 5%. (a) Find the annual premium. (b) Find the reserve at time 2, assuming the process is at that time in (i) state 0, (ii) state 1, (iii) state 2. 19.5 Verify that the Kolmogorov forward equations withs= 0 give Equation (8.15) in the case of a single life, or Equation (11.10) in the multiple decrement model. 19.6 Derive the Kolmogorov backwards equations. For timess≤ t, 𝜕 𝜕spi,j(s,t) =− N∑ k=1 𝜇i,k(s)pk,j(s,t). 19.7 Redo Examples 19.8 and 19.12 now given that𝜇01 = 1, 𝜇02 = 3, 𝜇10 = 2, 𝜇12 = 3. 19.8 For a two-state chain, derive the matrix (19.15) directly by using eigenvalues and eigenvectors. 19.9 In a two-state model, the forces of transition are given by𝜇01(0,t) = 1 for alltwhile 𝜇10(0,t) = { 1i f 0 ≤ t< 1, 3i f 1 ≤ t. Find the probability that the process will be in state 0 at time 2.4 given that it is in s t a t e1a tt i m e1 . 4 . 19.10 A stationary Markov chain has the following transition matrix. P = ⎛ ⎜ ⎜ ⎝ 0.60 .10 .3 0.20 .50 .3 0.10 .30 .6 ⎞ ⎟ ⎟ ⎠ . Interestisaconstant5%.FindtheAPVofthefollowingcontractsbasedonthischain, both issued when the process is in state 0. (a) An temporary annuity provides 100 per year, beginning when the process enters state 1 and stopping when the process leaves state 1. Payments do not resume upon re-entry to state 1. The last possible payment is at the end of 4 years. (b) A 7-year term insurance pays 100 at the end of the year of thefirsttransfer from state 0 to state 2, provided that that prior to this time there was a transfer from state 1 to state 0. If a transfer from state 0 to state 2 occurs before a transfer from state 1 to state 0, then nothing is paid. EXERCISES 331 19.11 Consider the chain of Figure 19.1. Suppose that (x) andyare two independent lives, where x is subject to a constant force of mortalitya and (y) is subject to a constant force of mortalityb. (a) Write down the reduced intensity matrixMr, (as defined in Example 19.8). (b) Writedownthereducedtransitionmatrix Pr(t)directlyfromChapter10formulas. (c) Find the eigenvalues and eigenvectors ofMr. (d) Verify that the rows ofPr(t) are linear combinations of vectors of the form e𝜆t(a1,a2,a3)where 𝜆isaneigenvalueof Mr and(a1,a2,a3)isthecorresponding eigenvector. 19.12 Consider the chain of Figure 19.1. We do not necessarily assume independence. You are given the constant intensities𝜇01 = b, 𝜇02 = a, 𝜇13 = c, 𝜇23 = d. (a) Given any positiveu, v, find the probability that (x) survives u years and (y) survivesvyears, in terms ofa,b,c,d. (b) Show thatT(x) andT(y) are independent if and onlya= candb= d. 19.13 Consideramulti-statemodelforthethreelives( x),(y),(z)wherewehaveeightstates described below, where the stipulated lives are those still living, State0:all;State1:(x ),(y)only;State2:( x),(z)only;State3:( y),(z)only;State4: (x) only; State 5: (y) only; State 6: (z) only; State 7: None. You are given that𝜇01 = 0.02, 𝜇02 = 0.01, 𝜇03 = 0.04, 𝜇34 = 0.02, 𝜇35 = 0.06, 𝜇36 = 0.05,𝜇67 = 0.10 and the force of interest is a constant 0.05. (a) Find the actuarial present value of an insurance policy on the three lives which pays1atthemomentofdeathof(z)providedthat(x )diesfirstand(y)diessecond. (b) Find the probability that the lives die in the order (x),(y),(z). 19.14 A contract is based on a three-state chain. There are benefits at the end of the year of transfer, of 100 for transfer from state 0 to state 2, and 50 for transfer from state 1 to state 2. Annual premiums of 10 are paid while the process is in state 0. There are annuity payments of 5 made each year when the process is in state 1. You are given thati 5 = 0.10 and the transition matrix P5 = ⎛ ⎜ ⎜ ⎝ 0.60 .10 .3 0.20 .70 .1 0.200 .8 ⎞ ⎟ ⎟ ⎠ . If 5V(0) = 50, 5V(1) = 35 5V(2) = 70, find 6V(i),f o ri= 0,1,2. 19.15 Redo Example 19.16 with the following changes in the given data: (i) The transition matrix for the year running from time 5 to time 6 is now given by the matrixP1 of Example 19.1. 332 MULTI-STATE MODELS (ii)Thereisapaymentattheendoftheyearof20000whenahealthylifebecomes unhealthy, and the yearly income for unhealthy lives is reduced to 15 000. 19.16 A insurance on two independent lives (x) and (y) provides for a death benefit at the moment of the second death. The death benefit ise0.05t for death at timet.P r e m i u m s are payable until the second death, The premium is level while both are alive and reduces to one half of the initial amount upon the first death. The life (x) is subject to aconstantforceofmortalityof0.10and( y)issubjecttoaconstantforceofmortality of 0.15. The force of interest is a constant 0.10. Refer to the two-life model as given Figure 19.1. (a) Using the method in Chapter 10, calculate for each of the first three states the annual premium and the reserve at timet. (b) Write down the Thiele differential equations. (c) Verify that your answers to (a) satisfy the equations in (b). 19.17 In a certain four state chain, the forces of transition are given by 𝜇 01(t) = 0.1+0.02t, 𝜇12 = 0.2+0.01t, 𝜇03(t) = 0.1+0.03t, 𝜇13 = 0.2+0.02t. For all otheri,jwithi≠ j, 𝜇ij(t) = 0. The force of interest is a constant 0.06. The following double integral represents the net single premium for a certain insurance contract. ∫ 3 0 ∫ t 0 e−(0.6+0.09t+0.02s)(0.1+0.02s)(0.2+0.01t)dsdt. Describe the benefits on the contract. 19.18 Prove Theorem 19.4 by observing first that pii(s,t+h) = pii(s,t) pii(t,t+h). 20 Introduction to the Mathematics of Financial Markets 20.1 Introduction This chapter will introduce some basic concepts of modern mathematical finance. One goal is to cover the fundamentals of option pricing. This has become an important tool in actu- arial mathematics, since many insurance and annuity contracts today contain the so-called ‘embedded options’, which we discussed in Section 13.2. For the most part, we carry this out in a discrete setting, but we do move into the continuous-time approach briefly in order to introduce the Black–Scholes–Merton formula. Another major objective in this chapter is to revisit the basic quantity of a discount function which we introduced early on. In the first part of the book, we treated this as a deterministic function, but a more realistic approach wouldbetoconsider v(s,t)asarandomvariable,reflectingthestochasticnatureofinvestment returns that we discussed in Chapter 14. In particular, we seek a version of the key identity, Formula (2.1) in this stochastic setting. A prerequisite for this chapter is the starred Section 2.12. We assume familiarity with concepts discussed in that section such as as short selling, forward contracts, and arbitrage. 20.2 Modelling prices in financial markets A financialmarket is an institution designed to facilitate the trading of financial assets, such asstocksorbonds.Certainindividualswishtobuysuchassets,whileotherswishtosellthem, and the financial market provides a forum to bring together the various parties. The potential holderofanassetquotesadesiredsellingprice,knownasthe askingprice,whilethepotential buyer quotes a desired buying price, known as thebidding price. When bidding and asking prices are equal it establishes a price for the asset, and a sale can be made. Our goal in this section is to develop a stochastic model for the evolution of prices. FundamentalsofActuarialMathematics , Third Edition. S. David Promislow. © 2015 John Wiley & Sons, Ltd. Published 2015 by John Wiley & Sons, Ltd. Companion website: http://www.wiley.com/go/actuarial 334 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS For most of this chapter, we assume a relatively simple framework, which will allow us to present the main ideas without too many technical mathematical difficulties. We assume a discrete-time model. That is, trading of assets will take place at integer times 0,1,2,…. The timeperiodcanbearbitrary,andwecanthinkofitaspossiblyaveryshorttime,(sayanhouror evenaminute)whichcouldthenconstituteanapproximationtothemorerealisticcontinuous- time setting. We adopt a finite-time horizon, with timeNas the last date we are interested in. Finally, we assume that the price of any asset can take on only finitely many possible values. Suppose we haveM+1 assets traded in our market, numbered from 0 toM. We will let S j(n) = the price of thej-th asset at timen. Weconsidereachsuchpriceasarandomvariable.Therefore,ourfinancialmarketismodelled byM+1 discrete time stochastic processesS j(n), wherej= 0,1,2, …M, andn= 0,1, …N. Wewillsingleoutaparticularasset,oftenreferredtoasa bankaccount,forassetnumbered 0.Todescribethis,wewillfirstneedtopostulateinourmodelanonnegativequantity rcalled the risk-free rate of interest, which is the interest rate that we can obtain on a risk-free investment as described in Section 2.12. We can then define this asset by S 0(0) = 1, S0(n) = (1+r)n. Inotherwords,thisisanassetwhichaccumulatesattherisk-freeinterestrate.Itisastochastic processinwhicheachrandomvariabletakesasinglevaluewithprobability1.Forsimplicity, we are at first adopting a constant risk-free interest rate. The definition could be based on a more general discount function and we comment on this below in Section 20.13. We will make the same idealized assumptions that we made in Section 2.12. That is, we postulate that for each asset, any real number of the units can be bought at any trading date. Throughshortsellingifnecessary,thisincludesnegativequantities.Wealsoassumethatthere are no transaction costs such as commissions. Notethattheexistenceofthebankaccountmeansthatweareassumingthatallparticipants in our market can freely borrow at the risk-free rate. Another simplifying assumption made throughout is that none of our assets provide any payments at intermediate dates, such as dividends on stocks or coupons on bonds. They provide funds only upon sale or maturity. 20.3 Arbitrage An initial observation is that in the typical financial market, the various asset prices do not move independently. If asseti moves up in price, assetj may have a tendency to move up, or possibly to move down, or be certain to move up or down. It can be quite complicated to model all dependencies, but theno-arbitrageprinciple will often enable us to reduce the possibilities. In our stochastic models, this requires a more complicated definition than the one we gave in Section 2.12. We first define the concept of atradingstrategy. This is roughly a description you, as an investor in the market, would give to an assistant before leaving for a holiday on a remote desert island where you cannot be reached. You would specify the number of units of each ARBITRAGE 335 assettobeheldateachtimeperiod.Ateachtradingdateaftertheinitialportfolioisestablished at time 0, certain assets in the existing portfolio would be sold and others bought to achieve the stipulated amounts. These amounts could depend on the entire past and present history. The description could be enormously complicated or quite simple. For example, a trading strategy might be as follows: Start with an initial portfolio of 1 unit each of asset 1 and asset 2. Keep this intact until the first time that the price of asset 1 is above 40 per unit and the price of asset 2 is below 30 per unit. At that time, sell all units of asset 1 and use the proceeds to buy shares of asset 2. These are then held without further trading. Note that the amounts to be held at timen can depend on all the prices of all assets at or before timen, but not after. It would not be a feasible trading strategy to specify that a a certainassetshouldbesoldattime2,ifthepriceofsomeotherassetwerebelow40attime3. Toformalizethissomewhat,wecanrepresenttheassetholdingsatanytime nbyavector 𝛼(n) = (𝛼 0(n),𝛼1(n)…𝛼M(n)) where𝛼j(n) is the number of units of assetjheld at timen. The entries of this vector are random, depending on the prices up to timen. So a trading strategy is formally a vector of these random vectors.  = (𝛼(0),𝛼(1)…𝛼(N−1)), where each𝛼(r) is a function of the values ofSj(k)f o rj= 0,1, …M, k= 0,1,2 …r. For any trading strategy and any timen, we will have a portfolio consisting of a certain number of units of each of ourM+1 assets. The portfolio at timen will then then have a valueV(n)obtainedbymultiplyingthenumberofunitsofeachassetbythepriceofthatasset at timen, and summing. That is V(n) = M∑ j=0 𝛼j(n)Sj(n), a random variable depending on all prices as well as the trading strategy as followed up to time n. Of courseV(0) is a definite number as it is the cost of setting up the initial portfolio at time 0 when all prices are known. For any trading strategy, there is a reverse strategy which involves holding at each time, the negative of the number of units held in the original strategy. In other words, one sells in placeofbuyingandbuysinplaceofselling.Formally,ifatradingstrategyisgivenbythethe vector , the reverse strategy is given by− .I fV ∗ denotes values for the reverse strategy it is clear thatV∗(n) =− V(n) for alln. Here is another important concept. Definition20.1 Atradingstrategyissaidtobe self-financingifforanytradingdateaftertime 0andbeforetime N,thetotalpriceofalltheassetssoldonagiventradingdateexactlyequals the total price of all the assets bought on that date, so no additional infusion or withdrawal of capital is required. For a self-financing strategy, the value of the portfolio at any intermediate trading date is the same before and after trading. 336 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS Wecannowsummarizetheprocedurewewillbefollowinginsubsequentdiscussions.We set up an initial portfolio at time 0 for a cost ofV(0), dictate a self-financing trading strategy, retreat to the dessert island, where no additional outlays of cash are required and none are received.Finally,theportfolioisliquidatedattime Nforproceedsof V(N),arandomvariable which depends on both the trading strategy and the evolution of prices. We letPdenote the probability measure forV(N). Thekeydefinitionofthissectioncannowbegivenintermsofthestartingvalue V(0)and the ending valueV(N). Definition20.2 Thefinancialmarketadmits arbitrageifthereexistsaself-financingtrading strategy such that V(0) = 0, V(N) ≥ 0, and P[V(N) > 0] > 0. A financial market which does not admit arbitrage is said to bearbitrage-free. In other words, an arbitrage opportunity is one where starting with a zero investment, we cannot possibly lose by the end of the trading period, and we have at least some chance of makingagain.Notethatthearbitrageopportunitydoesnotguaranteeapositivegain.Onecan think of it as being given a lottery ticket for free. We cannot lose anything, and there is some chance of profiting. It is important to note that cases where there is a very small probability of loss do not constitute an arbitrage under this definition. The avoidance of the loss must be absolutely certain. Remark In the definition of arbitrage, we could replace the condition onV(N)b yV(N) ≤ 0 and P[V(N) < 0] > 0, since, if this holds, the reverse strategy will satisfy the original condition.Thislooksabitstrangeatfirst,butitsimplysaysthatifthereisastrategyforwhich we are sure not to gain, then the reverse strategy is sure not to lose. An important consequence of the above is the following. Theorem 20.1 In an arbitrage-free financial market, if there is a self-financing trading strategyforwhichV (n) isaconstantcforsomen,then c= V(0)(1+r) n. Proof. Modify the strategy by holding−V(0) units of the bank account at time 0, so that the new strategy has initial value 0. If necessary, modify the strategy further to stipulate that everythingshouldbesettledattime n,andtheproceeds(possiblynegative)lefttoaccumulate in the bank account at the risk-free rate until timeN. The new strategy will have the constant value ofc−V(0)(1+r) n at timen, and this must be equal to 0. If not, there would be a sure chance of having either a positive or negative amount at timeN, which would imply an arbitrage opportunity, by Definition 20.2 and the remark following this definition. Thiswasareasonablysimpleresult,butthereisanimportantmessagebehindit.Itsaysthat intheabsenceofarbitrage,ifwecanfindaself-financingtradingstrategywhicheliminatesrisk atsomepoint,thenourinitialinvestmentmustaccumulateattherisk-freerateuptothatpoint. OPTION CONTRACTS 337 Remark It is true that any trading strategy can be converted into a self-financing one by using the bank account. An excess of the sales over purchases can be placed in the bank account, while excesses of purchases over sales can be handled by borrowing. However this gives a different strategy with a different amount held in the bank account at timeN, and therefore a different value ofV(N). The self-financing hypothesis is therefore essential in the definition of arbitrage. For our first example, we consider a very simple financial market. We will takeN = 1, so atradingstrategyinvolvessimplyspecifyingtheinitialportfolio.Ourfinancialmarkethas,in addition toS 0, a single risky assetS1 consisting of a stock. We can assume, changing units if necessary,thatthepriceofaunitofthestockis1attime0.Supposethatthepriceofthestock at time 1 can only take two possible values,u or d (standing for ‘up’, ‘down’ respectively) with d< u, each with positive probability. We call this abinomial model to reflect the two possible values at time 1. Theorem 20.2 Theabovefinancialmarketisarbitragefreeifandonlyif d< (1+r) < u. (20.1) Proof. Consider any trading strategy withV(0) = 0. If𝛼 is the number of units of stock in the initial portfolio. we must have−𝛼 units of the bank account. Then we will have either V(1) = 𝛼u−𝛼(1+r)o rV(1) = 𝛼d−𝛼(1+r). Suppose (20.1) holds. If𝛼< 0 then the first such value will be negative and the second will be positive, while the reverse holds if𝛼> 0. If𝛼 = 0, both values are 0. An arbitrage opportunity cannot exist. Conversely if (20.1) is not true, then at least one of two possibilities holds. Suppose d≥ (1+r). We create an arbitrage opportunity by choosing𝛼> 0 which makes both values of V(1) nonnegative,with at least one positive. The other possibility is thatu≤ (1+r), in which case we similarly create an arbitrage opportunity by taking𝛼< 0. Note that the converse statement is intuitively obvious. If the inequality does not hold, then we can create an arbitrage opportunity by either buying a stock which is sure to yield more than the risk-free return, or short selling a stock which is sure to yield less than the risk-free return. Anotherpertinentfacttonoticeinthedefinitionofarbitrageisthattheconditiondoesnot dependontheparticularvaluesof Pbutonlyonwhethersuchvaluesarepositiveorzero.We are therefore led to make use of the following standard definition of probability theory. Definition 20.3 Two probability measuresP and Q on a sample spaceS are said to be equivalent if for allA⊆ S,w eh a v eP(A) = 0 if and only ifQ(A) = 0. (For readers familiar with the concept of an equivalence relation, one can readily verify that this is a legitimate such relation.) It is clear from the definition that a financial market is arbitrage-free with respect toPif and only if it is arbitrage-free with respect to any equivalent probability measureQ. 20.4 Option contracts Given a financial market, we can do more than just buy or sell the existing assets. We have already seen one possibility, which is to enter into forward contracts. Another possibility 338 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS is option contracts.These are in one sense similar to forward contracts since they are both transactions which involve trading of assets at a future date for prices that are specified now. There are major differences however, for in an option, unlike the forward contract, one party isnotobligatedtocompletethetransaction,buthastheoptiontodoso,andwillonlyexercise thisoptionifitisadvantageous.Therearetwobasictypesofoptioncontracts,knownas calls and puts. A buyer of a call option has the right to buy a specified asset at a specified future time, known as theexpirationdate orexercisedate, for a specified price, known as thestrike price or exercise price, if they should choose to do so. The call option buyer has a similar motivation to a speculator taking a long position in a forward contract. They hope for a rise in price, so that they can buy the asset at a price which is lower than prevailing at the time of purchase. If the price of the asset at the expiration date is below the strike price, the option will not be exercised. A buyer of a put option has the right to sell a specified asset on the expiration date for a specified strike price. The put option buyer has a similar motivation to the speculator taking a short position in a forward contract. They hope for a fall in price so thattheycanselltheassetformorethanitisworthatthetimeofsale.Inthiscase,iftheprice of the asset at the expiration date is above the strike price, the option will not be exercised. Unlike the forward contract, the call and put buyers are not on opposite sides. For each of them, there must be another party who sells or (as it is commonly said) writes the option, and agrees to complete the transaction should the option holder so elect. Now if the option is exercised,theoptionwriterisnecessarilysellingorbuyingatanunfavourableprice,andthey are compensated for this by the option price which they receive from the option holder at the time the agreement is entered into. The option writers of course hope that options will not be exercised, so they profit by the full amount of the option price, and do not have to engage in an unfavourable transaction. Determining option prices is complicated, and will form much of the material of this chapter. It should be noted that what we have described are more properly known asEuropean options, which specify that the option can only be exercised on the one specific expiration date. We will assume all options we discuss are of this nature unless specified otherwise. Another type of contract, known as anAmericanoption, allows for the exercise of the option at any time before or on the expiration date. These are more complicated and will be dealt with briefly in Section 20.7. Althoughtheunderlyingassetsforcallsandputarenormallytakenasfinancialinstruments like stocks, as will be the case in our treatment, the basic idea of an option arises in many diversecontexts.Forexample,buyinginsuranceonanassetlikeahouse,isessentiallybuying a type of put option. You are protecting yourself from a drop in value, not from market variationinthiscase,butratherfromphysicaldamage.Similarly,theguaranteesforvariables annuities (as discussed in Section 13.2) which protect your account against unfavourable investmentexperience constituteputoptions.Foranotherexample, supposethatyoutakeout a long-term loan or mortgage, and the lender gives you the right to repay in full at any time without penalty. In effect, you have been given a call option. In this case, you are protected from a rise in the cost of repayment, which will occur if interest rates decline. (Refer to the discussion in Section 2.10.3.) Inessence,protectionagainstdeclinesinthevalueofanassetthatyouown,whileallowing youthefullbenefitofincreasesinvalue,canbeviewedasbeinggivenaputoption.Protection against increases in the value of an asset that you may wish to acquire in the future, while allowingyouthefullbenefitofdecreasesinvalue,canbeviewedasbeinggivenacalloption. Note that in contrast, forward contracts protect you from unfavourable declines or increases, OPTION PRICES IN THE ONE-PERIOD BINOMIAL MODEL 339 but they do not allow the parties to reap the full upside benefits, since the transactions must be completed with the agreed upon prices. 20.5 Option prices in the one-period binomial model In this section, we show how the no-arbitrage principle allows us to calculate an option price for the one-period binomial market where condition (20.1) holds. We illustrate with a particularexample.Supposethatashareofthestockissellingfor108attime0andattime1it willbeeither132or99,eachwithpositiveprobability.Assumearisk-freeinterestrateof10%. Consider a call option on the stock with an expiration date of time 1 and a strike price of 110. What should the price per unit of this option be? At first, one may think that there is no way to determine this exactly, and that it could take on many possible values. After all, the option is just another asset with its price being determined by the amounts bid and asked by the various market participants. The worth of this asset, however, is directly tied to the performance of the stock, so it should be clear that its price must be related in some way to the stock price. Such an asset is often termed aderivativesecurity, since its value is derived from that of another security. Tohelpdeterminetheprice,wetakethefollowingpointofview.Purchasersofcalloptions arenotnormallyinterestedinactuallytakingpossessionofthestockatmaturity.Theysimply wanttobuyitatthestrikeprice,andsellitimmediatelyforthehighermarketpriceifavailable. If the market price is below the strike price, the option is worthless and they receive nothing. The option then is just another assetS 2 with S2(1) = 132−110 = 22 if the stock price goes up, orS2(1) = 0 if the stock price goes down. The problem is to determineS2(0). Thosewell-versedintheactuarialmodelswediscussedinearlierchaptersmaywellthink that we can determineS2(0) by simply taking a discounted expected value, as we did with several other similar sounding problems. That is, we simply take the price as 22vpwherevis thediscountfactorforoneperiod,and pistheprobabilitythatthestockgoesup.Wewillfirst illustrate why one cannot solve the problem this way, and after that, we will, paradoxically, illustrate why onecando it this way. The first problem is that one is not givenp as part of the model. All that we postulated abouttheprobabilitymeasure Pwasthatbothofthepossibleoutcomesattime1havepositive probability.Indeed,theremaynotbeanyreasonablechoiceforasinglevalueof p.Theman y different participants in the market may well have completely different assessments of this figure. It is not unusual to find two experts commenting on a particular stock, where one claimsitisthebestbuyingopportunitytocomealonginthelastdecade,andtheotherpredicts imminent bankruptcy of the firm. The second problem is that one is not givenv. Now the reader may take issue with this statementsincewepostulatedariskfreerateof10%afewparagraphsback,soitappearsasif v issimply(1 .10) −1.Useofthisratewouldimplythatthebuyerislookingforanexpectedreturn of 10%on their investment. However, 10% is the return for a perfectlyrisk-freeinvestment. Investing in a call option is far from being risk-free. If the stock price at expiry is below the strike price, the entire investment is lost. It is to be expected that a rational option purchaser will want a return in excess of 10% as compensation for taking on the risk. (Recall that we discussed the same concept when introducing the risk discount rate in Section 12.4). We will now solve the puzzle, and show that regardless of the assessment ofpor of the desired yields of different individuals, the price of this option can only be 12. The reason is 340 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS that one can in factreplicatethe option for an initial investment of 12. That is, by investing only in the bank account and the stock, one can produce an outcome at the expiration date, which matches exactly the payouts of the option. This is done by buying 2∕3o fas h a r eo f stock at time 0, which will cost 72. We can put in 12 cash, and borrow the additional 60. If the stock price is 132 per share at time 1, we sell our 2∕3 of a share for 88, pay off the loan balance which is now 66, leaving us with 22. If the stock price is 99, we sell our 2∕3o fa share for 66, and pay off the loan, leaving us with nothing extra. We have therefore exactly replicated the option for the price of 12. It is is clear that no one would pay more than 12 to buy this option. Similarly, nobody would sell the option for a price of less than 12, since instead they could reverse the above strategy and be in the same position at time 1 as if they had written the option, but they would have have received 12 at time 0. Here is another point of view, which ties in with our previous definition of arbitrage. If we enlarge our financial market by adding the option as another assetS 2, then we must takeS2 = 12tomakethisenlargedmarketarbitrage-free.Totakeadefiniteexample,suppose the option price is 13. We will construct an arbitrage opportunity. Take the trading strategy which has as initial portfolio𝛼(0) = (−59,2∕3, −1). The reader can verify thatV(0) = 0. Now S 0(1) =− 64.90, so IfS1(1) = 132, thenS2(1) = 22 andV(1) = 1.1. IfS1(1) = 99 then S2(1) = 0 and againV(1) = 1.1. We leave it to the reader to find an arbitrage opportunity if the option price is below 12. Let us now go back to the proposed solution of of 22vpas an option price, which we criticized a few paragraphs above. If we in fact use the risk-free rate and therefore take v= 1.10 −1, we will get the correct answer by usingp= 0.6. Is there someway we could have discovered this probability of 0.6 beforehand? The answer is yes. Let us suppose that thereexistsasocalled risk-neutralindividual,thatisonewhoignorestheriskandishappyto acceptanexpected10%returnonanyinvestment,regardlessofthedegreeofsafetyinvolved. Letpbetheparticularprobabilityofriseinthestockprice,whichwouldbeassumedbysuch a risk-neutral person. In order that this person would be willing to pay 108 for a share of stock, we should have that 108 = 1.10 −1[132p+99(1−p)] and solving we have indeed thatp= 0.6. Wehavenowdiscoveredtheimportantprincipleof risk-neutralvaluation.Theassignment here of 0.6 and 0.4 to the events of the stock going up or down, respectively, is known as a risk-neutralprobability measure. It is the probability that must be assigned by a risk-neutral individualinordertojustifybuyingthestockatthemarketprice.Notethatwearenotsaying that such a person necessarily exists, and indeed have stressed that most investors would be unlikely to possess such an attitude. We are only saying that if one did exist, the price of the underlying asset would necessarily imply a unique probability assessment for that individual. The principle then says that if we use the risk-free interest rate, along with the risk-neutral probability measure, then we can indeed value options by following the usual actuarial approach of taking a discounted expected value. Note carefully that the risk-neutral probability measure need not be the same or indeed haveanyparticularrelationtotheoriginalmeasure P,otherthanbeingequivalentinthesense defined above. Even if we had specified values forP, these would have had no effect on the resultingoptionprice.Thefactthatonecanrisklesslyreplicatetheoptionmeansthatonlythe risk-neutral probability and the risk-free interest rate need be considered. OPTION PRICES IN THE ONE-PERIOD BINOMIAL MODEL 341 This is a puzzling observation at first, and for those who are still skeptical, we will look into the situation a little further. We mentioned above that the probabilityp was not even specified as part of the model, but let us suppose it is. In fact, suppose that instead of a stock with uncertain returns, we have two lotteries each depending on the same random draw. A ball is drawn randomly from an urn containing two white balls and one red. The payoff from lottery 1 at time 1 is 132 if a white ball is drawn, or 99 if a red is picked. The payoff from lottery 2 at time 1 is 22 if a white ball is drawn or 0 if a red is drawn. So the true underlying value ofpisnowindisputableas2∕3.Ifthepriceforalottery1ticketis108,andwemakethe assumptionthatwecanbuyorsellanyfractionoflottery1tickets,thenthepriceforalottery 2 ticket must be 12, by exactly the same argument as given above, regardless of the known value ofp.Whatdoesthisimplyforpeoplewhoparticipate?Buyersofaticketinlottery1are in effect earning an expected return of [(2∕3)132+(1∕3)99)]∕108−1 = 12.04%. There is a reasonable extra return over the risk-free rate, to compensate for the risk taken on. Buyers of a ticket in lottery 2 are in effect earning an expected return of [(2∕3)22∕12]−1 = 22.22%, a much higher return, which compensates for the greater risk in lottery 2 when the entire stake couldbelost.Indeedforanyvalueof pabove0.6,therewillbeareturnabovetherisk-freerate inlottery1andanevenhigherreturninlottery2.Itisonlyfortherisk-neutralvalueof pequal to 0.6, for which the expected returns on both lotteries will coincide with the risk-free rate. Going back to our original example with the stock, is it possible that an investor who assesses the probability of an upward movement as being less than 0.6 would still pay 108 per share, thereby earning an expected return of less than the risk-free rate? This may seem irrational, but it is no more so than the behaviour of a vast number of people who buy lottery tickets or gamble in casinos at highly unfavourable odds. (For more on this topic, see Example 22.2.) Wenextderiveageneralformula.Supposethat(20.1)holds.Aswedidabove,wecanset up an equation to solve forp, the risk-neutral probability that the upward move will occur. TakingS 1(0) = 1, this is 1 = (1+r)−1[pu+(1−p)d], (20.2) which we solve to obtain p= (1+r)−d u−d ,1 −p= u−(1+r) u−d . (20.3) Note that condition (20.1) ensures that 0< p< 1. The above procedure allows us to uniquely price, not only call options, but a general derivative security in this market, which pays an amountA if an upward move occurs orB if a downward move occurs. We do this in one of two ways: first, we can find a replicating initial portfolio consisting of𝛼 units of the stock and𝛽 units of the bank account by solving the equations 𝛼S 1(0)u+𝛽(1+r) = A, 𝛼S1(0)d+𝛽(1+r) = B. (20.4) Then Price = 𝛼S1(0)+𝛽, (20.5) which is the cost of establishing the replicating portfolio. Secondly, and usually easier, we can bypass finding the replicating portfolio and just take the price as the discounted expected 342 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS value of the the payoff with respect to the risk-free interest rate and risk-neutral measure. That is, Price = (1+r) −1[pA+(1−p)B], (20.6) where p is as given in formula (20.3). The reader can verify that both methods lead to the same answer. Example 20.1 For the example given at the beginning of this section, find the price of a put option with a strike price of 110. Solution. If the upward move occurs, the holder tears up the option. If the downward move occurs,theholderbuysthestockfor99,andsellsitfor110.Sothisisaderivativesecuritywith A= 0,B= 11.DirectlyfromEquation(20.6),wehavethatthepriceis1.10 −1(0.4×11) = 4. Alternatively, solve (20.4) to derive the replicating portfolio given by𝛼 =− 1∕3,𝛽 = 40, and use (20.5) to get the same answer. To see this directly, we replicate the option for a cost of 4 byselling1∕3ofashareshort,receiving36,lettingthetotalof40accumulateto44attime1. This allows one to just cover the short position if the stock is up, or cover the short position and have 11 left over if the stock is down. There is in this case yet another way to obtain the answer. In fact, we develop a general formula relating puts and calls. Theorem 20.3 (Put-call Parity) Let 𝛾 denote the cost of a call option and𝜋 denote the cost of a put option on the same stock with a current price of S(0), the same strike price of K,andsameexpirationdateN.Then S(0)+𝜋 −𝛾 = K(1+r) −N. (20.7) Proof. Suppose an investor at time 0 adopts the following trading strategy. Buy one unit of stock, sell one call option, buy one put option, and hold these without further trading up to theexpiration dateN.If S(N) > K,theput willexpire worthless,thecallwillbeexercised by the other party, so that the investor must give up the stock for a price ofK.I fS(N) < K,t h e callwillexpireworthless,theinvestorwillexercisetheputandsellthestockforapriceof K, while ifS(N) = K, both options are worthless and the value is just the stock price. Whatever happens, the value at time N of the portfolio will beK. SinceV(0) is just the left side of Equation (20.7), the formula follows from Theorem 20.1. The proof shows in fact that this theorem is true for a general arbitrage-free market, and does not depend on the binomial assumption. In our present example, we know𝛾 = 12, N = 1, S(0) = 108 andK = 110, and we can immediately calculate that𝜋 = 4. 20.6 The multi-period binomial model The model of the last section is clearly too simple to be representative of reality. As a further extension, we keep the binomial feature, but allow the prices to evolve over several periods. THE MULTI-PERIOD BINOMIAL MODEL 343 We assume that the price of the stock evolves each period as described in the one-period model above. That is,ifthevalue issat thebeginning of aperiod, the priceat theend willbe eithersuorsdwhered< u.Soforaninitialpriceof S(0)attime0,thepriceattime1willbe either S(0)uor S(0)d, and the price at time 2 will be eitherS(0)u2,S(0)ud,o rS(0)d2,e t c .W e can represent this by what is known as abinomialtree. See, for example, Figure 18.1 which is an example withu= 1.2,d = 0.8. We can now consider any general contingent claim, which will be a payoff at timeN which can depend on the entire history of up and down movements in the stock price. To formalize this, consider the sample spaceΩ consisting of all paths in the binomial tree. Each such path can be labeled by anN-termed sequence formed of the entries U and D, where U denotes an upwards branch and D a downwards branch, so there are 2 N paths altogether. We postulate that there is a probability measurePon Ω, but we need not specify anything about it except thatP(𝜔) > 0 for all𝜔 ∈Ω . We now formally define the general type of derivative security we are interested in. Definition 20.4 Acontingentclaim is a contract which provides a payment at timeNwhich is dependent on the particular outcome in our underlying sample space. It is modelled by a random variableX, where for𝜔 ∈Ω ,X(𝜔) is the payment for outcome𝜔. Forexample,acalloptionwithstrikeprice Kandexpirationdate Nonastockwithcurrent price S(0) is a contingent claim given by X(𝜔) = [S(0)u md(N−m) −K]+ whenever𝜔isasequencewith mupwardmovementsand N−mdownwardmovements.(For any real numbert, the symbolt+ denotes max{t,0}.) A contingent claim can be more complicated than the options we have described up to now. Consider, for example, alookback option on a stock which will return at expiry the maximum value of the stock over the period from time 0 to timeN. So looking for example at Figure 18.1, we would have X(DUD) = 96, X(UDD) = 120, and so on. Themulti-periodmodelhasthesameessentialfeaturesthatweobservedintheone-period model. r The financial market consisting of the stock and the bank account is arbitrage-free if and only if condition (20.1) holds. r Any contingent claim can be priced uniquely so as to prevent arbitrage. One method is to find a replicating self-financing trading strategy. The price of the claim is then the cost of setting up the initial portfolio for this strategy. A second way is to take the expected discounted value with respect to therisk-neutral probability measureQ on Ω,whichissimplythemeasureobtainedbyapplyingtheappropriateprobabilities por 1−pas given by formula (20.3) to each branch of the binomial tree. That is, if𝜔 has mentries of U andN−mentries of D, Q(𝜔) = p m(1−p)N−m 344 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS These facts can be verified by using the results from the one-period model, and working backwards in time. The definition of contingent claim gives us directly its value at timeN. We use those to determine the value and strategy applicable to each node at timeN−1, and then use these to get determine the value and strategy applicable to each node at timeN−2, and continue to iterate the procedure until we get to time 0. Example 20.2 Consider a two-period model where the price of a stock evolves as shown by the tree in Figure 18.1 up to time 2, andr= 0.10. The contingent claimXis a call option at time 2, with a strike price of 92. So X(UU) = 52, X(UD) = X(DU) = 4, X(DD) = 0. Find the replicating strategy and the price of the option which will prevent arbitrage, Solution. Supposeattime1,thevalueofthestockis120.Weknowthat u= 1.2,d= 0.8,and we can solve the system (20.4) withA= 52,B= 4t og e t𝛼 = 1,𝛽 =− 920∕11. This means that if the process is in the upper node at time 1, then in order to replicate the payoff at time 2,weshouldown1unitofstock,andcarryadebtof920 ∕11.Thetotalvalue V(1)is400∕11. Similarly,ifthevalueofstockis80attime1,wesolvethesystem(20.4)with A= 4,B= 0, andwearriveatarequiredportfolioof1 ∕8unitsofstockandadebtof80∕11foratotalvalue V( 1 )o f3 0∕11. Wenowmovebacktotime0andagainsolvethesystem(20.4)with A= 400∕11and B= 30∕11,toobtainainitialportfolioconsistingof37 ∕ 44sharesofstockandadebtof7100∕121. The value of this initial portfolio isV(0) = 3075∕121 which must be the price of the option. To summarize, one can replicate this contingent claim by the following self-financing tradingstrategy.Attime0,buy37∕44unitsofthestock,using3075 ∕121ofone’sowncapital andborrowingtheremaining7100∕121attherisk-freerate.Attime1,ifthestockmovesup, increase the stock holding to 1 unit, borrowing additional funds to do so. If the stock goes down, sell enough to reduce the stock holding to 1∕8 unit, using the proceeds to partially repay the loan. This replicating strategy gives in addition ahedgingstrategy. Suppose you have just sold such an option. You run the risk that the stock will move up both periods. If you do not actually own the stock, you will be required to buy it at 144 and sell it at 92 (which shows the danger of selling a so-callednakedoption on a stock you do not own). If you follow the trading procedure outlined above, you will be sure to be able to meet your obligation in any event, assuming of course that the given model for the evolution of the stock price is correct. Tocalculateonlytheoptionprice,ratherthanthecompletereplicatingstrategy,thesecond method can be used. That is option price= (1+r) −N [ ∑ 𝜔∈Ω X(𝜔)Q(𝜔) ] . (20.8) For the particular case of a call option with strike K, this takes the form option price= (1+r)−N [ N∑ m=0 ( N m ) pm(1−p)N−m(S(0)umdN−m −K)+ ) ] , (20.9) THE MULTI-PERIOD BINOMIAL MODEL 345 where p is determined by formula (20.3). In our casep= 3∕4, and we can verify that, as before, the price is (1.1)−2[52(9∕16)+4(6∕16)+0(1∕16)] = 3075∕121 Example 20.3 In the example given above, suppose the interest rate is 0. Find a price and self-financing trading strategy for the so-calledlookback option, which pays at timeN the maximum value of the stock at time 0,1,2. Solution. The payoff is 144 for the outcome UU, 120 for the outcome UD, and 100 for each of the outcomes DU and DD. We can find the price exactly as we did for the option above. Forr= 0 we can calculatep= 1∕2 and Price = [144(1∕4)+120(1∕4)+100(1∕2)] = 116. To find the trading strategy, we need a more complicated diagram. See Figure 20.1. In the original diagram (Figure 18.1), the paths of UD and DU both led to the same position at time 2. This is fine in cases where the prices of the stock at that point was all we were interested in, since this took the same value of 96 on both paths. However, in this case, the contingent claim is path-dependent and we need two different nodes to distinguish the two paths. Fortheuppernodeattime1,weneedtohold 𝛼 unitsofstockand 𝛽 unitsofbankaccount where 144𝛼 +𝛽 = 144, 96𝛼 +𝛽 = 120, so that 𝛼 = 1∕2 𝛽 = 72, V(1) = 132. For the lower node at time 1 we similarly solve 96𝛼 +𝛽 = 100, 64𝛼 +𝛽 = 100. so that 𝛼 = 0, 𝛽 = 100 V(1) = 100. 100 120 80 96 144 64 96 Figure 20.1 Example 20.3 346 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS We are then back to a one period model with an asset that has value 132 if the upward movement occurs, or 100 if the downward movement occurs. So the initial portfolio must have𝛼 units of stock and𝛽 units of the bank account where 120𝛼 +𝛽 = 132, 80𝛼 +𝛽 = 100 so that 𝛼 = 0.8, 𝛽 = 36, V(0) = 116. Sothetradingstrategyistostartwith116(asweknewfromthefirstsolutionabove),buy 0.8 units of stock and put the rest in the bank account. If the upward move occurs, sell 0.3 unitsofstock,orifthedownwardmoveoccurs,sellall0.8unitsofstock,ineachcaseputting the proceeds into the bank account. For contingent claims which depend only on final prices, the first type of diagram, (like Figure 18.1) known as arecombiningtree, provides a significant reduction in computation. This was not readily apparent in our simple example whereN = 2, but suppose instead that N = 10. The recombining tree would have 11 final nodes, while the more general version would have 2 10 = 1024 final nodes. 20.7 American options WeincludeabriefdiscussionhereonAmericanoptions,whichcanhavesomesurprisingand initially puzzling features. Recall that such an option can be exercised at any trading date up to and including the final expiration date. One’s intuition tells us that the price of this should begreaterthanthepriceofthecorrespondingEuropeanoption,sincethereismorechoiceand thereforeachanceformorepotentialgain.Howeverone’sintuitionisnotcompletelycorrect. Infact,givenourassumptions,anAmericancalloptionshouldneverbeexercisedpriortothe expiration date, so in fact the two options are equivalent and should bear the same price. If r= 0, the same phenomenon holds for an American put option. It is never correct to exercise early. However, in the more usual case whenr> 0, it may well be correct to exercise the put option early. We now clarify these rather curious facts. Consider a particular example. You hold an American call option to buy an asset for a strikepriceof100andonacertaintradingdate n< N,theassetpriceis300.Yourdesiretotake advantage of this high price might induce you to exercise the option, making an immediate profit of 200. After all, at a later date the asset price could be lower, with a corresponding reduced gain. However, one should not exercise the option, since there a better way to take advantage of the higher price. You just sell the asset, relying on the option to protect you against further increases, the usual danger with short sales. By doing this and waiting, you would receive 300 immediately. In the worst case scenario, you then have to buy the asset at maturity for 100 and settle your short position. But your gain as of the expiry date would be 200 plus the interest earned on the entire 300 that you received at timen, in addition to an extra gain if the price is below 100 on the expiry date. If you exercised the option early, your gain at expiry would be limited to 200 plus the interest earned on the 200 received at timen. Note that this conclusion depends heavily on our reasonable assumption thatr> 0 and is not true for negative interest rates. AMERICAN OPTIONS 347 Our argument also does not apply to dividend-bearing stocks, since it is possible that by exercisingearlyandreceivingthestock,thedividendspaidwillmorethancompensateforthe loss of interest. It does show however that the only possible times when one should exercise are those which coincide with dividend payments. Exercising before such a date will at least incur the loss of interest up to the dividend payment date. We will not go into the complete analysis in this case. Now consider an American put option. Suppose now that at timenthe strike price is 300 and the price of the asset is 100. Should one take advantage of the low price, by buying at 100 (assuming you don not already own the asset), then exercising the put to sell at 300 and makinganimmediateprofitof200?Iftheinterestrateis0,theanswerisno,becausesimilarly to the call option case there is a better way to take advantage of the current low price. In this reversed situation, we simply borrow 100 and buy the asset, relying on the put to protect us again future drops in the price. At expiry, we sell the asset for a minimum of 300, repay the loan, and have a gain of at least 200. So again with an interest rate of 0, the American and European puts are equivalent. But consider the more realistic case of a positive interest rate. Ourpreviousargumentdoesnotholdnow,sincebywaiting,wearepayingoutinterestrather thanreceivingitasinthecaseofacalloption.Supposethatinanyevent,wedecidetoborrow 100 at timento buy the asset, and the interest charged over the period fromnto N is 5%. If we exercise immediately, we receive 300, which increases with interest to 315 by expiry, and afterrepaymentoftheloan,ourgainattime Nis210.Ifwewaittoexercise,wewillbebetter off if and only the asset price is higher than 315, in which case we keep the asset and tear up the option. So it is not immediately clear whether to exercise or not. In our discrete model, we can effectively work out the price and trading strategy for an American put option by the same backwards induction process that we illustrated in Examples 20.2 and 20.3. One simply must do an extra comparison at each node. Suppose we have calculated data for all nodes at times greater thann and we are considering a node at time n. One first works out the strategy and a temporary valueV exactly as in the European case. One then compares that value with what could be obtained from immediate exercise at time n. This is calculated by buying (or selling) a sufficient quantity of the asset so that you hold one unit and then selling that unit for the strike price. If this exercise value is greater thanV, then that replacesV as the value, and the strategy is to exercise at that node. The following is a simple one-period example, which is sufficient to illustrate the tech- nique, since, in all cases, you just follow the procedure below at each node. Example 20.4 An asset sells now for 100, and at time 1, will have a price of either 120 or 80, both with positive probability. The risk-free interest rate is 0.10. Find as a function ofK the price of an American put option with a strike price ofK. Compare this with the price of a corresponding European put if (i)K = 113 and (ii)K = 102. Solution. The value at time 1 is (K−120) + for an upward move and (K−80)+ for a downward move. By (20.3), the risk-neutral probability of the downward move is 1∕4. In the extreme case thatK ≤ 80, the option is clearly worth nothing. Take the other extreme where K ≥ 120. The value at time 0 in the European case would be [(3∕4)(K−120)+(1∕4)(K− 80)](1.1 −1) = K(1.1)−1 −100 which is less thanK−100. The price of the option isK−100 and the strategy is to exercise immediately at time 0. 348 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS Now consider the case when 80< K < 120. The price will be the maximum of {K−100, 0.25(K −80)(1.1)−1} where thesecond term isthepriceof theEuropean put. Wecan solve toshow that immediate exercise is optimal precisely when K ≥ 1800∕17 = 105.88. So forK = 113 the price of the American put is 13 as compared to 7.50 for the European put. WhenK = 102, the price of both options is 5. 20.8 A general financial market We often wish to model situations which are much more complicated than the ones we consideredintheprevioussection.Foronething,wemayhaveseveralriskyassetsratherthan one. For another, the evolution of prices may be given by a more involved structure than the binomial tree, and even in the binomial case, it may have a more complicated form than the constant up and down ratios ofuandd. Atypicalexampleofsuchageneralmarketismodelledasatreelikeevolutionwhichwill apply to all assets. See for example Figure 20.2 At each timenwe have a number of nodes, which we can think of as representing a certain ‘state of nature’, and all the asset prices are determined by this state. This market has three assets and the prices (S 1(n),S2(n)) are shown at each node. The asset 0 prices need not be shown as they are known completely once we specifyr. We will now describe the general discrete-time model. The notation is necessarily some- what involved, but the market of Figure 20.2 is sufficient to capture the main ideas. We have a finite state stationary Markov chain (see Section 18.2) with the following special structure. The set of states is divided up into subsets 0,1 …N, wherek denotes the set of states at timek. There is a single states0 ∈ 0 reflecting the fact that there is no uncertainty about prices at time 0. The only transitions of positive probability are those from a state to another one which is one period later. That is, given statesiin  k and jin m where mis notk+1, we must havepij = 0. For example, in our multi-period binomial model, there werek+1 states at each timek, and for each state at timek, there were exactly two transitions into a state at timek+1. In the market of Figure 20.2, we would have0 = 0, 1 = {1,2,3}, 2 = {4,5,6,7,8,9,10}. In the binomial case, we described outcomes by sequences consisting ofUand D.I nt h e general case, where we can have more than two branches from a node, we need a somewhat different representation. The evolution of our system up to timek can be described by a sequence of states {s 0,s1,…,sk} where eachsk ∈ k and psj,sj+1 > 0f o rj< k. We will call such a sequence anadmissiblek-sequence. We now take our sample spaceΩto consist of all admissible N-sequences, and this is equipped with the probability measurePwhere for any admissible N sequence 𝜔,w eh a v eP(𝜔)=Π N−1 k=0 psk,sk+1. As an example, in the market of Figure 20.2,Ω consists of the 7 elements {{0,1,4}, {0,1,5},{0,2,6},{0,2,7},{0,3,8},{0,3,9},{0,3,10}}. A GENERAL FINANCIAL MARKET 349 120, 130 State 1 100, 90 State 2 100, 100 State 0 130, 160 State 4 110, 100 State 5 105, 100 State 6 95, 80 State 7 90, 100 State 8 80, 70 State 9 70, 70 State 10 80, 80 State 3 Figure 20.2 A finanical market with two risky assets It is convenient to adopt the following notational device. For any admissiblek-sequence 𝜈 withk< Nwelet v◦ denotethesetofall 𝜔 ∈Ω whichextend 𝜈.Thatisif v= s0,s1,…,sk, then v◦ = {𝜔 ∈Ω: the first k+1 entries of𝜔 are s0,s1,…sk,}. (20.10) For example, in the market of Figure 20.2, {0,3}◦ will denote the subset {{0,3,8},{0,3,9}, {0,3,10}}. 350 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS Wenowwanttocaptureformallythenatureofquantitieslikeassetpricesorassetholdings atacertaintime k.Thesearerandombeforetime k,butarethenknownwithcertaintyattime korlater.Forexample,inthemarketofFigure20.2,thepriceofasset1attime1isarandom variable which is uncertain at time 0, but then is known precisely at time 1. So it will take the constant value of 120 on the set {0,1} ◦ = {{0,1,4},{0,1,5}}. Similarly it will take the constant value of 100 on the set {0,2}◦ and the constant value of 80 on the set {0,3}◦. We handle this in general by the following definition. Definition 20.5 For any integerk= 1,2, …,N, a random variableV defined onΩis said to be k-determinedif V is constant on any set𝜈◦ where𝜈 is an admissiblek−sequence. For ak-determined random variableV, we will writeV(𝜈) to denote the constant value of V on the set𝜈◦. Itfollowsfromthedefinitionthata k-determinedrandomvariableisalso m-determinedif m≥ k. This just reflects the fact that if we know something at timek, we will know it just as well at some later time. Note that a 0-determined random variable is just a constant, since the only 0-admissible sequence is that with the single entry ofs0 ands◦ 0 is the entire setΩ. Definition 20.6 For any random variableW, we define ak-determined random variable Ek(W) as follows. Suppose that the point𝜔 ∈ 𝜈◦. That is,𝜈 comprises the beginningk+1 entries of𝜔. We then define Ek(W)(𝜔) = E(W|𝜈◦) using the notation of (20.10). So Ek(W) then just gives the expected value ofW conditional on the firstk-steps of the evolution. Itisclearfromthedefinitionthat Ek(W)isk -determined.E0(W),being0-determined,isa constant and just equal to the usual expected valueE(W). Consider the other extreme where k= N.Thenforany 𝜔,theset𝜈 ◦ isthesinglepoint 𝜔andE(W|𝜔)isjustW (𝜔),showingthat EN(W) = W.S oa skincreases,Ek(W) gives us more and more information aboutW until we reach timeN and knowW exactly. Example20.5 ConsiderFigure20.2.Supposethatwheretherearetwobranchesemanating from a node, the probability of an upward move is 2∕3 and that of a downword move is 1∕3, whileinthecaseofthreebranchesemanatingfromanode,eachhasprobability1∕3.Describe the random variableE 1(S1(2)). Solution. Consider the setB= {0,1}◦, which consists of the two points, namely {0,1,4} that has probability 4∕9 and {0,1,5} that has probability 2∕9. We could calculate that the conditional probabilities givenB are (4∕9)∕(6∕9) = 2∕3 for {0,1,4} and (2∕9)(6∕9) = 1∕3 for{0,1,5}.Observenowthatwedidnothavetodoallofthiscalculation,sincethetree-like structuremakesitpossibletoreadofftheseconditionalprobabilitiesfromthefuturebranches of the tree without worrying about the past. In this case, with only one future step, they are immediate. We then have thatE 1((S1(2)) takes the value of 2∕3(130)+1∕3(110) = 123 1∕3 on{0,1,4}and{0,1,5}. Similarly,ittakesthevalueof1012∕3on{0,2,6}and{0,2,7}and the value of 80 on {0,3,8},{0,3,9} and {0,3,10}. ARBITRAGE-FREE CONDITION 351 To summarize, a financial market withM+1 risky assets and of durationN is modelled by aMarkov chain withthe special structureas noted above, a probability measure onthe set Ω of all paths from time 0 to timeN, a risk-free interest rater, and random variablesS j(n), j= 0,1, …M, n= 0,1, …,N,o n Ω where each Sj(k)i s k-determined. A trading strategy consists of a collection of random variables𝛼j(n), j= 0,1, …M, n= 0,1, …N−1 where each 𝛼j(k)i sk-determined. For an important application to follow, we now turn to the concept of amartingale introduced in Section 18.3, and look at conditions for this to occur in our present context. Fix a probability measureQonΩwhich is equivalent toPand let {W n},n= 0,1, …,N be a sequence of random variables such that eachWk is k-determined We claim that this will be a martingale, provided Ek(Wk+1) = Wk, k= 0,1, …,N−1. (20.11) To see this, suppose that the above holds. Fix anyk and a sequence of real numbers {w0,w1,…,wk}. Consider any set which has positive probability underQand is of the form A= {𝜔 ∈Ω: Wi(𝜔) = wi,i= 0,1, …,k}. Nowbydefinition,membershipin Aisdeterminedbywhathappensuptotime k.Ifasequence 𝜔 ∈ A, any sequence which has the same firstk+1 entries must also be inA. This implies thatAmustbetheunionofsubsetsoftheform 𝜈◦ forsome k-admissiblesequence 𝜈. Forany such𝜈, it follows from our hypothesis (20.11) that E(Wk+1|𝜈◦) = Ek(Wk+1)(𝜈◦) = Wk(𝜈) = wk and from (A.22), (applied to the sample sample spaceA with the conditional probability Q(⋅|A) we can conclude that E(Wk+1|A) = wk, showing that the sequence is a martingale. To apply this, refer again to Figure 20.2. Let Q be the probability measure which assigns 1∕3toeachtransitionwhentherearethreetransitionsoutofastateand1∕2toeachtransition whentherearetwo.Wecanthenseethatthesequenceofpricesofasset1isamartingaleunder thismeasure,bysimplyverifyingthecondition(20.11)ateachnode.Forexample,atstate3,we havethatthevalueof S 1(1) = 80andthevalueof E1(S1(2)) = (1∕3)90+(1∕3)80+(1∕3)70 = 80. The same holds at all other states. Similarly, we can show that the same holds forS2(n), the sequence of prices of asset 2. 20.9 Arbitrage-free condition To decide when a general financial market is arbitrage-free, directly from Definition 20.2, couldbeextremelycomplicated.Wewouldhavetoconsiderallpossibleinitialportfolioswith value 0 and all possible self-financing trading strategies. Fortunately, there is often a faster way. Suppose we can find a probability measureQ, equivalent toP, such that for each asset i, the sequence {S i(n)} is amartingaleunderQ. Consider any self-financing trading strategy. 352 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS At any timen, the portfolio has a valueV(n). For each asseti, the expected value at time n+1 will again beSi(n) and so the expected value of the portfolio before trading will be V(n). Since our trading strategy is self-financing, the expected value after trading will again be V(n). Since this is true for all possible values of the portfolio at timen,w em u s th a v e that EQ[V(n+1)] = EQ[V(n)]. (The subscript indicates that expectations are with respect to the probability measureQ.) Working inductively, we have thatEQ[V(N)] = V(0) = 0. It is impossible forV(N) to be nonnegative for all outcomes, have a positive probability of being positive, and still have an expectation of 0, so we cannot have an arbitrage opportunity. This seems like a nice simple answer but on the face of it there is a major problem. It is not reasonable to expect that our stochastic processes for stock prices are martingales, as we indicated in Section 18.3. In fact, the bank account, by definition, cannot be a martingale unlessr= 0.Soourresultabovemayappearatfirsttobemeaningless,butthefollowingtrick saves the day. We do not have to measure our assets in terms of dollars. They can be expressed relative to some other asset. Define ̂S j(n) = Sj(n)∕S0(n). That is,̂Sj(n)i st h ev a l u eo fa s s e tjat timenin terms of the bank account. We can think of ̂Sj(n)asadiscounted orpresentvalue,sinceitiswhatwewouldhavetoinvestinourrisk-free bank account in order to accumulate toSj(n) at timen. It is a random variable rather than a number sinceSj(n) is a random variable. The same argument we gave above clearly goes through if eacĥSj(n) is a martingale. This is now possible sincêS0(n) takes a constant value of 1. We have therefore proved the ‘if’ direction of the following major result. Theorem 20.4 (The fundamental theorem of asset pricing) A financial market is arbitrage-free if and only if there is a probability measure Q onΩ which is equivalent to P,andforwhich ̂S j isamartingaleforj = 1,2, …,M. A major example is the the multi-period binomial model, where the given risk-neutral measure satisfies the conditions of the above theorem, as we verify from Equation (20.11). Indeed, suppose that̂S(k) = s, so thatS(k) = s(1+r) k. Referring to Equation (20.3), for any 𝜔 ∈Ω , Ek(Sk+1)(𝜔) = s(1+r)k u[(1+r)−d)+d[u−(1+r)] u−d = s(1+r)k+1 so that Ek[̂S(k+1)] = s. As another application, we can conclude immediately from our observations in the pre- ceding section that the market of Figure 20.2 is arbitrage-free whenr= 0. Note that the risk-neutral probabilityp that we gave in the one-period binomial market was the only possible value that would makêS1 a martingale, as shown by Equation (20.2). The terminology is carried over and any probability measureQsatisfying the conditions of Theorem 20.4 is known as arisk-neutralmeasure. The main conclusion of this section then is that usually the best way to show a given a given financial market is arbitrage-free is to show the existence of a risk-neutral measure. The converse of the fundamental theorem will be proved in the following section. EXISTENCE AND UNIQUENESS OF RISK-NEUTRAL MEASURES 353 20.10 Existence and uniqueness of risk-neutral measures 20.10.1 Linear algebra background To complete our study of financial markets, we require a knowledge of some facts in linear algebra. We assume familiarity with the concept of a linear space (also known as a vector space) and linear subspaces. We also assume familiarity with the concepts of closed and bounded sets. Any basic text on multivariate calculus should contain the necessary details. The following is a brief review, adapted to our ultimate goals. Consider in particular the vector spaceW consisting of all real-valued functions defined on some finite setS, with the operations of point-wise addition and scalar multiplication. This is ann-dimensional space wheren is the number of points inS. We let 0 denote the function which takes the value 0 at each point ofs. (The context should distinguish this from the number 0.) For anyf,ginW, we have an inner product f ⋅ g= ∑ s∈S f(s)g(s). A subsetKof W is said to beconvexif it contains the line segment joining any two of its points.Thatis,given f andginKandandascalar0 <𝛾< 1,thefunction 𝛾f +(1−𝛾)gisin K. AhyperspaceinWisaproperlinearsubspaceofmaximumdimension,thatisonelessthan thedimensionofthespace.Soforexample,ahyperspacecanbevisualizedintwo-dimensional spaceasalinethroughtheorigin,orinthree-dimensionalspaceasaplanethroughtheorigin. We need the following two facts about hyperspaces. The first is a fairly standard result and not difficult to verify. The second is quite a bit more advanced. 1. Anyhyperspace Hisdeterminedbyitssocalled orthogonalvector.Thatis,thereisan elementq≠ 0i nW such that H = {h∈ W : q⋅ h= 0}. The elementq is unique up to a scalar multiple. In two or three dimensions, we can visualize it geometrically as a vector perpendicular toH. 2. LetLbeanylinearsubspaceof Wandlet Kbeaclosedandboundedconvexsetthatdoes notintersect L.Thenthereahyperspace HcontainingLsuchthat Kdoesnotintersect H. It is simple enough to visualize this geometrically in three-dimensional space. If a linedoesnotintersectaclosedandboundedconvexset,wecanfindaplanecontaining the line which does not intersect the set. This of course does not hold if the set is not convex. Suppose thatK is a doughnut-shaped region, and the line goes through the hole. Then any plane containing the line must intersectK. 20.10.2 The space of contingent claims We return now to our model as described above and apply our linear algebra concepts. For a given financial market, define the following sets. Let W be the linear space of all real valued functions onΩ. 354 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS We can view this as the space ofcontingent claims, those payments at timeN which are determined by the particular path. An important subspace ofW is given by L= {f ∈ W : there exists a self-financing trading strategy such that for all𝜔 ∈Ω , f(𝜔) = V(N)(𝜔)}. So L is the subspace of allreplicable claimsas defined above in Section 20.5. It is a linear subspace since, givenf and g in L, we can replicatef +g by just holding at each stage the sumoftheholdingsinthetradingstrategiesreplicating f andg,andwecansimilarlyachieve any scalar multiple off by multiplying our holdings by that scalar. Let L0 = {f ∈ L: there is a replicating self-financing trading strategy forf with V(0) = 0 This easily seems to be a linear subspace ofL.W el e t K = { f ∈ W : f(𝜔) ≥ 0 for all𝜔 ∈Ω , ∑ 𝜔∈Ω f(𝜔) = 1 } . which is a convex subset ofW. So a nice linear algebra definition for a financial market to be arbitrage-free is to simply say thatL0 does not meetK. (Of course any nonzero, nonnegative functionf inL0 represents an arbitrage opportunity, but an appropriate scalar multiple of such anf will be inKand also in the subspaceL0. We also use the fact that our original probability measurePmust take a positive value on each𝜔.) To illustrate, Figure 20.3 gives a geometric picture of the one-period binomial market. Any contingent claim is represented by a point (f(U),f(D)) in the plane. The setKis the line f(D) L0 ‐ arbitrageL0 ‐ arbitrage-free (0,1) K (1,0) f(U) Figure 20.3 A picture of the one period binomial market EXISTENCE AND UNIQUENESS OF RISK-NEUTRAL MEASURES 355 100 120 80 100 Figure 20.4 A market in which not all contingent claims are replicable segmentjoiningthepoints(0,1)and(1,0).Thesubspace L0 isapropersubspaceandtherefore must be a line through the origin. In the arbitrage-free case, this line will have negative slope and not meetK. In the case of an arbitrage opportunity,L 0 as represented by the dotted line, has a slope that is either nonnegative, or equal to∞, and it must intersectK. The picture also makesitclearthat Listheentireplane,aswenoticedenough,sinceitasubspacethatproperly contains L0. Further examples are furnished by the markets of Figures 20.4, 20.5 and 20.6. Assume that r= 0. Alternatively, we can assume any positiverand interpret the asset values that are given as aŝSj(n) rather thanSj(n). The conclusions will be the same in either case. In the single risky asset market of Figure 20.4, the setΩ will have three pointsU,M,D (for‘up’,‘middle’,‘down’).Iftheinitialportfoliohas 𝛼 unitsofstockand 𝛽 unitsofthebank account, the time 1 value of portfolio will be 120𝛼 +𝛽 for the upward movement, 100𝛼 +𝛽 if the price stays the same, or 80𝛼 +𝛽 for the downward movement. It follows that L= {f : f(U)+f(D)−2f(M) = 0}, atwo-dimensionalsubspaceof W,showingthatnotallcontingentclaimsarereplicableinthis market.Inparticular,acalloptionwithstrikeprice110willhave f(U) = 10,f(M) = f(D) = 0 which is not inL. For initial portfolios of value 0, we must have in addition that 100𝛼 +𝛽 = 0 leading to L0 = {f : f(M) = 0:f (U)+f(D) = 0}. 100, 100 130, 120 60, 80 Figure 20.5 A market that is not arbitage-free 356 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS 111, 91 120, 100 90, 70 110, 90 Figure 20.6 See Exercise 20.7 The intersection ofL0 and K is clearly 0, showing that this market is arbitrage-free. Of course we could have immediately deduced this from the Fundamental Theorem, since assigning probabilities of 1∕3 to each branch yields a risk-neutral measure. Consider the two risky asset market of Figure 20.5. An initial portfolio with value 0 will be given by the vector of the form 100(−(𝛼 +𝛽),𝛼,𝛽). Then, a functionf will be inL 0 if we can find𝛼,𝛽 satisfying 30𝛼 +20𝛽 = f(U), −40𝛼 −20𝛽 = f(D). (20.12) These equations are readily solved to give 𝛼 =− (f(U)+f(D))∕10,𝛽 = (4f(U)+ 3f(D)∕20.Itfollowsthat L0 = L= W.Sothismarketisaboutasfarfrombeingarbitrage-free as we could possibly get. Any contingent claim can be replicated for an initial cost of 0! Obviously, the prices here as shown could not be maintained by rational investors. To see the delicacy of situations like this, look at this market again, but make the modifi- cationthat S 1(1) = 70insteadof60.Weleavetothereadertoverifythatwestillhave L= W, but L0 is quite different. The coefficient of𝛼 in the first equation of system (20.12) is 40 instead of 30, so that now L0 = {f ∈ W : f(U)+f(D) = 0 as in the market of Figure 20.3, which shows that the market is arbitrage-free. We can also deduce this immediately from the Fundamental Theorem, since now there is a risk-neutral measure, Q(U) = Q(D) = 1∕2. The market of Figure 20.2 that we investigated in Section 20.8 is more complicated, and it would involve a great deal of calculation to try to deduceL 0 exactly, although we do know thatitcannotmeet Kduetothearbitrage-freecondition.Itispossibletoshowthat L= W,but thisisfarfromobviousfromthefiguresasgiven.Asaparticularcase,considerthefollowing. Example 20.6 Let X be the contingent claim that that takes the value 60 on UU and 0 elsewhere. Taker= 0. Find a self-financing trading strategy to replicate this claim. Solution. For any such strategy, each of the lower two nodes at time 1 will lead to a claim of 0 at time 2, so by the martingale property, our portfolio must have value 0 at these nodes. Therefore, looking at the pre-trading values at time 1, 𝛼 0(0)+100𝛼1(0)+90𝛼2(0) = 0, 𝛼0(0)+80𝛼1(0)+80𝛼2(0) = 0, EXISTENCE AND UNIQUENESS OF RISK-NEUTRAL MEASURES 357 which gives 𝛼1(0) =− 0.5𝛼2(0), 𝛼0(0) =− 40𝛼2(0). Similarly,thevalueattheuppernodeattime1mustbe60timestheprobabilityofanupward move, which is 60(1∕2)= 30. So 𝛼 0(0)+120𝛼1(0)+130𝛼2(0) = 30, and substituting from above we have, 𝛼0(0) =− 40, 𝛼1(0) =− 0.5, 𝛼2(0) = 1. To summarize, the trading strategy at time 0 is to buy 1 unit of asset 2, financing this by selling 1∕2 unit of asset 1, borrowing 40 and putting up the remaining 10. This checks out since the initial cost must be 60Q(UU) = 60(1∕3)(1∕2). At time 1, if the middle or lower branchoccurs,selltheunitofasset2,whichisjustenoughtocovertheshortpositionandpay off the loan. Wemustnowdecidewhattodoattime1iftheupperbranchoccurs.Inthiscase,wehave 𝛼 0(1)+130𝛼1(1)+160𝛼2(1) = 60, 𝛼0(1)+110𝛼1(1)+100𝛼2(1) = 0. Thereareseveralsolutionstotheseequations,whichindicatesthatareplicatingself-financing trading strategy need not be unique. One example is to take 𝛼 0(1) =− 100, 𝛼1(1) = 0, 𝛼2(1) = 1. This strategy involves borrowing an additional 60 to cover the short position in asset 1. At time 2, we pay off the loan of 100, and have either 60 or 0 left, depending on what happened to asset 2 at time 2. Similarly, for each of the other six paths, we could find the replicating strategy for a claim which pays off only on that path. We would take a suitable linear combination of these seven strategies to replicate any possible contingent claim. This will show thatL= W.I n Section 20.11, we will prove a result which provides a much easier way to see this. 20.10.3 The Fundamental theorem of asset pricing completed In this section, we prove the converse to the result established above, and show that in any arbitrage-free financial market, wecan find arisk-neutralmeasureQ. We willproceed intwo stages. Stage 1:Defining Q: Bythearbitrage-freeassumption, L 0 doesnotmeettheset K,aconvex,closedandbounded set. By our linear algebra results of Section 20.10.1 we can find a hyperspaceH containing L0 and not meetingK.L e tqbe an element orthogonal toH. That isH = {h: q⋅ h= 0}. Nowitcannotbethatfortwodistinctpoints f andginK,weha v eq⋅ f < 0and q⋅ g> 0, for if so we could find𝛾 such that the functionh= 𝛾f +(1−𝛾)g satisfies q⋅ h= 0 and so 358 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS h∈ H. But by convexityh∈ K, and this would contradict the fact thatH does not intersect K. So by a change of sign, if necessary, we can assume thatq⋅ f > 0 for allf ∈ K.N o wi n particularthefunctions1 𝜔 whichtak ethev alueof1on𝜔 andthevalue0elsewherearein K, and so we can infer that for all𝜔 ∈Ω , q(𝜔) = q⋅ 1𝜔 > 0, and by multiplying by a suitable scalar we can ensure that ∑ 𝜔∈Ω q(𝜔) = 1, whichmeansthatthefunction qistheprobabilityfunctionforaprobabilitymeasure QonΩ. Stage 2:Showing the martingale condition: Fixany j.Wewillshowthatthestochasticprocess ̂Sj isamartingaleunder Q.Foranytime n< N and any possible values of Sj(n), letA denote the event thatSj(n) = s, which means that ̂Sj(n) = s(1+r)−n. We must show that EQ(̂Sj(n+1)|A) = s(1+r)−n, or equivalently that EQ(Y|A) = 0, (20.13) where Y = ̂Sj(n+1)−s(1+r)−n. Consider the following trading strategy. Do nothing before timen. If the price of assetj at time nis not equal tos, do nothing at all. If the price at that time iss, buy 1 unit of assetj, borrowing to do so, and sell it at timen+1. Apply the proceeds to repaying the loan and let thedifference(whichcouldbenegative)accumulateinthebankaccount.Let f bethefunction inW corresponding to this strategy. If Sj(n) = s, this strategy yields a bank account of [Sj(n+1)−s(1+r)] at timen+1. Multiplying by (1+r)N−n−1, the accumulated amount at timeN is (1+r)NY if the purchase is made and 0 if the purchase is not made. Our trading strategy is self-financing and requires an initial investment of 0. Therefore, the functionggiven by g(𝜔) = { (1+r) nY(𝜔)i f 𝜔 ∈ A 0, if 𝜔 ∉ A is inL0. This means that 0 = q⋅ g= ∑ 𝜔∈A q(𝜔)(1+r)NY(𝜔) = (1+r)NQ(A)EQ(Y|A). The fact thats is a possible value ofSj(n)i m p l i e st h a tQ(A)> 0, and so we must have EQ(Y|A) = 0, establishing Equation (20.13). COMPLETENESS OF MARKETS 359 One method of showing that a market is not arbitrage-free is to find the subspaceL0 and show that it intersectsK. But as we saw above, this can be computationally infeasible in all but the simplest cases. The converse of the Fundamental Theorem provides an easier way. Example 20.7 Use the above result to show that the financial market of Figure 20.5 is not arbitrage-free. Solution. Given a probability measure onΩ,l e tq be the probability of an upward move. For ̂S 1 to be a martingale, we need 130q+60(1−q) = 100 so thatq= 4∕7. For̂S2 to be a martingale, we need that 120q+80(1−q) = 100 so thatq= 1∕2. No such measure exists. 20.11 Completeness of markets Inthissection,weposethefollowingquestions.Givenanarbitragefreemarket,canwepriceall contingentclaimsbythetwomethodswehadinthebinomialmodel?Canwedosouniquely? Theuniquenessquestioniseasilyansweredforanarbitrage-freemarket.Asweshowedin thebinomialcase,ifwecanreplicateacontingentclaimwithaself-financingtradingstrategy, then the cost of that claim should be the costV(0) of setting up the initial portfolio. What happens, however, if there are several different replicating self-financing trading strategies? This can certainly occur, but inan arbitrage-freemarket, they necessarilyhave thesameV(0) whichmeansauniqueprice.Supposetothecontrarythatthereweretworeplicatingstrategies for the same contingent claim, one with an initial cost of 100 and the second with an initial cost of 60. The investor could follow both the second strategy and the reverse of the first strategy for a net gain of 40 at time 0 which would be placed in the bank account, resulting in an overall initial value of 0. At timeN, the payments on these two strategies would cancel, leaving a certain positive amount in the bank account, contrary to the fact that there were no arbitrage opportunities. We turn now to the existence question, beginning with a definition. Definition 20.7 A financial market is said to becompleteif, given any contingent claimX, there is a self-financing trading strategy that replicatesX. In other words, using the notation of the preceding section, the subspaceLis all ofW. We have already shown that the multi-period binomial market is complete. Moreover in Figure 20.4, we gave an example of an incomplete market. Thefollowingtheoremgivesacharacterizationofcompletenessforarbitrage-freemarkets. Theorem 20.5 An arbitrage-free market is complete if and only if there is a unique risk- neutralmeasure. Proof. Suppose that the market is complete. LetQ be any risk-neutral measure. Fix any 𝜔 ∈Ω .L e tX 𝜔 be the contingent claim that pays 1 if𝜔 occurs and pays 0 for all other outcomes, and choose a self-financing trading strategy that replicatesX𝜔.I fV(0) is the cost of the initial portfolio, the martingale property ensures that V(0) = (1+r)−NEQ(X𝜔) = (1+r)−NQ(𝜔), 360 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS so that Q(𝜔) = V(0)(1+r)N, showing thatQis uniquely determined. Conversely,supposethatthemarketisnotcomplete,sothat Lisnotequaltoallof W.W e can then choose a nonzero functionh∈ W such that h⋅ f = 0 for allf ∈ L. (20.14) (Since we can do this for a hyperspace, we can clearly do it for any proper subspace which is contained in some hyperspace.) The function which takes the constant value 1 is inL, (achieved by investing (1+r) −N in the bank account at time 0), so we must have that ∑ 𝜔∈Ω h(𝜔) = 0. (20.15) Let Q be the probability measure with the probability functionq as constructed in proving the ‘only if’ part of Theorem 20.4. We will produce a different functionq′ with the same properties asq, namely q′(𝜔) > 0, for all𝜔 ∈Ω , (20.16) ∑ 𝜔∈Ω q′(𝜔) = 1, (20.17) q′ ⋅ f = 0, for allf ∈ L. (20.18) Then q′ will induce a second martingale measureQ′. To constructq′, we note that since q(𝜔) > 0 for all𝜔, we can choose a positive number𝛿 sufficiently small so that the function q′ = q+𝛿h satisfies (20.16), and sinceh≠ 0, q′ is different fromq. In view of Equations (20.14) and (20.15), it is clear thatq′ also satisfies Equations (20.17) and (20.18). Toillustratetheproofofthelastpart,lookagainatthemarketofFigure20.3.Thefunction q(𝜔) = 1∕3 for all𝜔gives us a risk-neutral measure. The space L is, as we have seen, the set of all functions such that f(U)−2f(M)+f(D) = 0 and the perpendicular functionhcan be taken as h(U) = 1, h(M) =− 2, h(D) = 1. We can take then𝛿 to be any number strictlybetween−1∕3 and 1∕6, which yields an infinite number of risk-neutral measures. As a consequence of the above theorem, we can see immediately that the market of Figure 20.2 is complete, without going through the somewhat involved calculation of the replications that we did before. The probability assignment which we gave in Section 20.8 is clearly the unique risk-neutral measure. THE BLACK–SCHOLES–MERTON FORMULA 361 Incompleteness means that there are not sufficiently many assets to account for all the variations in possible contingent claims. Comparing Figures 20.4 and 20.5, we see that with three branches we need at least three assets in order to achieve completeness. This explains alsothefactthatwithonlyoneriskyasset,weneedabinomialmodeltoachievecompleteness. For an additional example, consider the two risky asset market of Figure 20.5. We leave it to the reader to decide whether or not it is arbitrage-free, and whether or not it is complete. Atthispoint,wesummarizeourconclusions.Supposewehaveanarbitrage-freefinancial market. If the market is complete, then for any contingent claimX, there is a unique price whichwillpreventarbitrageopportunities.Thiscanbefoundinoneoftwoways.First,choose a self-financing trading strategy to replicateX, and the price will beV(0). Second, take the discounted expected value of (1+r) −NX with respect to the unique risk-neutral measureQ. If the market is incomplete, then we can still find a unique price for the replicable contingent claims. However the non-replicable claims cannot be priced uniquely, since different choices ofarisk-neutralmeasurecangivedifferentresults.Someothercriteriamustbeusedtoarrive at prices. This does not mean of course that there are no restrictions on the price of such claims. Often, a range of values can be computed. Example 20.8 In the market of Figure 20.4, find the possible no-arbitrage prices for a call option with exercise date 1 and strike price 110. Solution. Add the option as another assetS 2 with a price of𝜋 at time 0. For a martingale measure which has probabilityp of an upward movement and probabilityq of staying the same,wehave120p +100q+80(1−p−q) = 100implyingthat2p +q= 1,sothat p< 1∕2. Applyingthemartingaleconditionforthenewassetgives 𝜋 = 10p,whichmeansthatwemust have 0<𝜋<5. Conversely, all such values are admissible since for any such𝜋 we obtain a martingale measure for the enlarged market by taking p= 𝜋∕10, q= 1−𝜋∕5. In some cases, specifying the price of certain contingent claims will determine others. To illustrate, having added the additional asset and having specified𝜋 in Example 20.8, the resulting martingale measure is unique, so we have in effect completed the market. The price specified for the option will determine unique prices for all other contingent claims. 20.12 The Black–Scholes–Merton formula The discrete-time model illustrates many of the fundamental principles of pricing contingent claims. There is a limitation, however. To be at all close to a realistic model, we would need an enormously large number of time periods and the computations will quickly become intractable.Thisisapparentjustfromlookingatthecasesfor N = 2oftheprevioussection.For actualcomputationandgreaterrealism,thepreferredmethodistousecontinuous-timemodels, which involve more advanced mathematical machinery. The relatively simple calculations effected by solving linear equations above will be replaced by solving differential equations. Sums of random variables are replaced by integrals of random variables, a concept that is technicallyverycomplex.Thecharacterizationsofarbitrage-freenessandcompletenesswhich we gave can be generalized to continuous-time settings but the proofs are far more difficult. A rigorous development of this is beyond the scope of the book, However we do want to 362 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS investigate theBlack–Scholes–Mertonformula. This is a pivotal result which in fact initiated much of the modern research into stochastic models in finance. In this section, it is convenient to use the risk-free force of interest𝛿 =l o g(1+r) instead ofr. We consider again the case of a financial market consisting of the bank account, and a single stock whose price at timetis S(t). The difference is that we now allow trading at any time,andmoreover,weallowassetpricestovarycontinuously.Wethereforemustmodel S(t) asacontinuous-timestochasticprocessandwillinfactchooseageometricBrownianmotion process, as given in Section 18.7. That is, for some constants𝜇 and𝜎, S(t) = S(0)e 𝜇t+𝜎B(t), whereB(t) is a standard Brownian motion. The problem is to find the price of a European call option with exercise dateN and strike price K. One approach is to make use of the results that we already know for the discrete- time model. Suppose we can find a probability measureQunder whicĥS(t) is a martingale. Then we can approximate our process with our discrete time multiple period binomial model with periods of length 1∕m, u= e 𝜎 √ 1∕m,d= e−𝜎 √ 1∕m and with risk-free force of interest= 𝛿∕m, provided we takemsufficiently large. This should seem plausible since the log of our binomial process is a random walk. In this discrete setting, the price of the option is given in Equation (20.9). Putting in all the parameters and taking limits asmgoes to∞,w ea r r i v ea t the celebrated Black–Scholes–Merton formula: Option price= S(0)Φ(𝛼 +𝜎 √ N)−Ke−𝛿NΦ(𝛼), (20.19) where 𝛼 = log(S(0)∕K)+(𝛿 −𝜎2∕2)N 𝜎 √ N , (20.20) and Φ is the c.d.f. of the standard normal distribution. An alternative method to derive the formulaistodirectlycalculate e−𝛿NEQ[(S(N)−K)+].Thisismorestraightforward,involving basiccalculus, although thecalculation issomewhat involved. For both ofthesemethods, we will omit the detailed derivations. Notethattheparameterstodeterminetheoptionpricearetherisk-freeforceofinterest 𝛿, the strike priceK, the durationN, and the quantity𝜎. The latter is a measure of the tendency for prices of the underlying stock to vary and is known as thevolatility of the stock. Note however that the formula does not depend on𝜇. We comment more on this below. The resulting formula seems somewhat formidable, but we can provide some motivation. Thefollowingisnotintendedasarigorousexposition,butmoretoprovideamethodofremem- bering and understanding the structure of the formula. First consider the random variable X =l o g[(S(N)∕S(0)]. What is its distribution under Q? From the definition of Brownian motion, we know that under our original probability measure, it is normal with mean𝜇N and variance𝜎 2N.I fw e make the assumption that it is still normal underQwith the same variance, and calculate the resulting meanM, we know from Equation (A.58) that EQ[S(N)∕S(0)] = eM+(𝜎2∕2)N. THE BLACK–SCHOLES–MERTON FORMULA 363 But, sincêSN is a martingale with respect toQ, S(0) = E(̂SN) = e−𝛿NE(SN), so that EQ[S(N)∕S(0)] = e𝛿N, leading to M = (𝛿 −𝜎2∕2)N. (20.21) Weconcludethatunder Q,wehaveanewdriftparameter(𝛿 −𝜎2∕2)whichiscompletely independent of the original drift parameter𝜇. This is analogous to the observation in the simpleone-periodbinomialmodelwheretheoriginalprobabilityplayednoroleintheoption price and it was only the risk-neutral probability that mattered. Now let us ask, what is the probability underQthat the buyer of this option will exercise it? This will occur ifS(N) > K. The probability of this is the same as the probability that X ≥ log(K∕S(0) which equals the probability that−X ≤ log(S(0)∕K), which by the results calculated above is Φ ( log(S(0)∕K)+(𝛿 −𝜎 2∕2)N 𝜎 √ N ) . (20.22) Given our above assumption regarding the distribution ofX, we have now identified the term Φ(𝛼) in the Black-Scholes-Merton formula as the probability under the risk-neutral measure that the option will be exercised. Now consider the following rather naive reasoning to arrive at the option price. If we exercise the option, we will have an expected gain at timeN of E[S(N)−K)] and this will have a present value ofE[S(N)−K]e−𝛿N. Multiply this by the probability of exercise to get S(0)Φ(𝛼)−Ke−𝛿(N)Φ(𝛼). This looks something like the actual Black–Scholes–Merton formula, but unless𝜎 = 0, the coefficient ofS(0) differs. Our naive approach, however, ignores the nature of the option contract. With positive values of𝜎, the stock price at timeNwill vary, and could be above or belowtheexerciseprice.Calculating E(S(N)−K)willincludenegativevalueswhentheprice isbelow K,butthereisnolossinthesecasessincetheoptionwillnotbeexercised.Thenaive formula therefore understates the true price. Another way of expressing this is to note that what we really want is the expected value ofS(N)−K, given that the option is exercised. To correct this understatement, it turns out that the coefficient of S(0) must be increased fromΦ(𝛼)t oΦ(𝛼 +𝜎 √ N). This makes sense as we would expect that this correction should increase as the variability in the stock price and length of period increase. For further insight and verification, we can directly verify the formula forN = 0. In this case, Φ(𝛼) =Φ (−∞) = 0i fS(0) < K, and the formula gives an option price of 0, which it should,while Φ(𝛼) =Φ (∞) = 1ifS (0) > Kandtheformulagivesanoptionpriceof S(0)−K which it should. There is still another important feature involving the two terms in the Black–Scholes – Mertonformula,whichwenowpresent.Startwiththefollowingquestion.Forthecalloption as above, what is its value at any timet,0 ≤ t≤ N? By the same argument as above, the 364 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS value will be given by the same formulas as above except that in both (20.19) and (20.20), S(0) is replaced by byS(t) which is the stock price at timetand N is replaced byN−t,t h e remaining duration. It turns out that the two terms of this formula yield a replicating strategy for the option, and therefore a hedging strategy, in terms of holdings of the stock and the bank account. To explain this, we will take new units for the bank account, by adopting a slightlydifferentbutequivalentpointofview,Wecanviewthebankaccountasaninvestment in risk-free zero-coupon bonds maturing at timeN. A unit of this asset will be a bond with face amount K, so that its value at time 0= K(1+r) −N.L e t𝛼t denote the value of (20.20) withS(0)replacedby S(t)and Nreplacedby N−t.Thereplicatingtradingstrategyistohold Φ(𝛼t +𝜎(N−t)) units of the stock, and carry a short position ofΦ(𝛼t) units of the bonds we just described, at any timet. At each timet, the value of this portfolio will be precisely the value of the option as given above and it will reach the right amount at timeN. In the case where the option is to be exercised the final portfolio will consist of 1 unit of stock, and a short position of 1 bond. In the other case, the portfolio will be reduced to zero units for both assets. It can be observed that the idea behind this strategy is similar to what we observed in thecalloptionofExample20.2,wherethereplicationisaccomplishedbybuyingstockasthe price increases, or selling as the price declines. One must also show that this strategy is self-financing, using an appropriate modification of this definition to apply to continuous time. The idea is that at each instant, the amounts received from selling one of the assets is exactly what is required to buy the other. A precise definition is based on the formulation given in Exercise 20.3, with the differences replaced by differentials. This is not easy, however. The problem is that these are differentials of stochastic processes rather than deterministic functions, and a rigorous presentation involves some knowledge of the subject known as stochastic calculus. Remark In real life of course this replication by continuous rebalancing is not possible. One could try to get close by very frequent rebalancing but there is no guarantee that this discrete-timeapproximationisself-financing.Itcouldwellrequireadditionalamountsofcash or release such. But one can hope that if our model is sufficiently accurate that these extra amounts are relatively small, sothat theoption could bereplicated forsomething closetothe Black–Scholes–Merton price. To conclude this section, we note that while the Black–Scholes–Merton formula has persistedasthemaintoolforoptionpricing,itisbasedonseveralsimplifyingassumptions.One such assumption is that of the log-normal evolution of stock prices. There has been evidence to show that this is not completely realistic, and alternative models have been investigated. Another assumption is that both the risk-free force of interest and the volatility are constant and known, as these must be inserted into the formula to obtain numerical results. More realistic models have been proposed where these quantities are both considered as random. 20.13 Bond markets 20.13.1 Introduction We now return to the discrete-time case. In this section, we deal with markets where the assetsarerisk-freezero-couponbonds.ThesewerealreadyintroducedinSection2.12,where BOND MARKETS 365 we considered forward prices, but here we want to consider the actual prices which will be random variables. A major result will be to give an appropriate version of 2.1 to ensure no arbitrage in the case of stochastic discount functions. Our market will consist ofN+1 assets where forn≠ 0, assetn is a zero-coupon bond maturing for 1 at timen. We will defineS 0 later. We will denote the random variableSk(n)b yv(k,n). Fork≤ n, v(k,n) is the price you would pay at timek to receive 1 at timen, which ties in well with our original use of this notation in Chapter 2, as well as our notation for forward prices. These are not the forward prices however, but the actual prices. Of coursev(0,n) is a real number and the same as the forward pricẽv(0,n), since at time 0 we know the prices of the bonds. But future prices are unknown and therefore modelled as random variables. Aquestion which may come tomind now iswhether the fundamental Equation 2.1could holdforthesequantitiesonlyinterpretedasamultiplicationofrandomvariables.Theanswer is no. Observe, for example, that with randomness v(0,m) ≠ v(0,n)v(n,m), since the left side is a real number while the right side is truly random. Now in order to avoid a completely trivial situation, we cannot assume a constant deter- ministicrisk-freerate raswedidbefore.Beforediscussinghowwemodifythisidea,wewant to recall how risk arises from investment in risk-free assets like bonds, (elaborating on the discussion at the end of Section 2.10.3). Note first that unlike a stock, where a value on any futuredateisunknown,thepayoffonarisk-freebondisabsolutelycertainifheldtomaturity. Theriskarisesifonewantstobuyorsellbeforethatdate.Forexample,iftherisk-freeinterest todayis5%,thenthepriceofa3-yearbondwillbe(1 .05) −3 = 0.864.Ifattime1,therisk-free interest rate has risen to 0.08, then the value of the bond drops to (1.08)−2 = 0.857. Buying long-termbondsthereforecarriestheriskofariseininterestrates,whichcanlowertheprice. However, in our present model, where trading occurs only at integer times, buying bonds whichmaturein1perioddoesnotcarryanyrisk.Thisleadsustodefinethebankaccountby S 0(0) = 1, S0(n) = [v(0,1)v(1,2) ⋯ v(n−1,n)]−1,n≥ 1. (20.23) Ourbankaccountisformedbystartingwith1attime0,usingthattobuyabondmaturing at time 1, then taking the proceeds ofv(0,1)−1 to buy a bond maturing forv(0,1)−1v(1,2)−1 at time 2, and to continue rolling over the account into a new 1-year bond each year. In the case of a constant and deterministic interest rate ofr, we would havev(k,k+1) = (1+r) −1, and this definition ofS0 reduces to the one given before. Note that under this new definition,S0(k) is not a definite amount, but strictly random for k> 1. Using the bank account, we can extend the definition ofv(k,n) to all ordered pairs (k,n) by defining v(k,n) = S0(k)∕S0(n), k> n, since this is the amount accumulated at timek, by taking 1 from a bond maturing at timen and placing it in the bank account. Note that with this definition, all prices at the final time N are determined by the previous values ofS 0(n) (and of course the fact thatV(N,N) = 1). 366 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS This means that we can model our bond market with a treegoing up to only timeN−1. (See Example 20.9 below.) 20.13.2 Extending the notion of conditional expectation For our analysis, we will need some additional results aboutk-determined random variables, We will in fact investigate this idea in more generality and further extend the discussion in Section A.8, since the same ideas are needed in Chapter 24. Suppose we are given a sample spaceΩ,aprobabilitymeasure PonΩandapartition Π={B 1,B2,…Bn}ofΩ intopairwise disjointsetswithunionequaltoallof Ω,suchthatforall i,P(Bi) > 0.Foranyrandomvariable W on Ω, define a random variableEΠ(W) as follows. For𝜔 ∈ Bi, EΠ(W)(𝜔) = E(W|Bi). Toillustrate,inthemodelofSection20.8,fix kandtakethepartition Πwhichconsistsof all sets𝜈◦ where 𝜈 is an admissiblek sequence. (For example, in the market of Figure 20.2 if we takek= 1, the partition will consist of the three sets {0,1}◦,{0,2} ◦,{0,3} ◦.) Then EΠ(W)i sj u s tEk(W) as defined in that section. We summarize the facts that we need in the following theorem. Theorem 20.6 TakeanyrandomvariablesV,W andascalarc. (a)EΠ islinear.Thatis EΠ(V+W) = EΠ(V)+EΠ(W), EΠ(cV) = cEΠ(V). (b)SupposethatW isarandomvariablewhichisconstantoneachsubsetof Π.Then EΠ(WV) = WEΠ(V). (c)Supposethat Π′ isa finer partitionthan Π whichmeansthateverysetin Π isaunion ofsetsin Π′.Then EΠ[E′ Π(W)] = EΠ′[EΠ(W)] = EΠ(W), Proof. (a)Thisfollowsdirectlybyapplying(A.23)and(A.8)toeachsubset Bi ofthepartition equipped with the probability measureP(⋅|Bi) (b) As in (a) apply (A.8) to each set of the partition. (c) It is clear from the definitions that if a random variable is constant on each subset of Π,thenapplying E Π toitwillleaveitunchanged.Now EΠ(W)isconstantoneachsubsetof Π therefore constant on each subset of the finer partitionΠ′. It follows thatEΠ′[EΠW] = EΠW. To show the other order of composition, choose any setAof the partitionΠ and letAbe the union of setsB1,B2,…Bm where eachBi is a set of the partitionΠ′. To simplify the notation denote EΠ(W)b yZ. Now by definition,Ztakes the constant value ofE(W|Bi) on each setBi so that clearly E(Z|Bi) = E(W|Bi). BOND MARKETS 367 Now apply (A.29) to the sample spaceAequipped with the probability measureP(⋅|A). We have that E(Z|A) = m∑ i=1 E(Z|Bi)P(Bi) = m∑ i=i E(W|Bi)P(Bi) = E(W|A) which shows that EΠ(Z) = EΠ(W) It is of interest to look at the extreme cases. If we takeΠ to the finest possible partition wherethesetsaresingletons,then EΠ(W)isjust W.Ifwetake Πtobethepartitionconsisting of just one set, namely the whole space, which is the least fine partition, thenEΠ(W)i sj u s t the usual expectationE(W). The latter observation leads to some results of interest which we use in Chapter 24. Immediately from part (c) of the Theorem 20.6, for anyW, E[EΠW] = E(W) (20.24) More generally, ifZ is any random variable which is constant on the sets of the partitionΠ, then from part (b) of Theorem 20.6 E(ZW) = E[EΠ(ZW)] = E[ZEΠW]. (20.25) 20.13.3 The arbitrage-free condition in the bond market Return to our bond market with the assumption that for allk, the random variablev(k,n)i s k-determined. What is the condition onv(k,n) to ensure that the market is arbitrage-free? Lets go back to the deterministic setting of Chapter 2 for a moment. Note that if a special case of Equation (2.1) holds, namely v(k,n) = v(k,k+1)v(k+1,n) (20.26) forallnonnegativeintegers k≤ n,thenastraightforwardinductionargumentshowsthat(2.1) willholdforallnonnegativeintegers k≤ m≤ n.(Example20.9belowwillmakeitclearwhy we begin with Equation (20.26) in place of the more general Equation (2.1)) We saw above that (2.1) does not hold when considered as a statement about random variables, but could it possibly be that this revised version will be valid? The answer is still no, since the left hand side isk-determined, but in general, sincev(k+1,m) is onlyk+1- determined, the right side will only bek+1-determined. It is plausible, however, that the following natural modification of our statement holds. Namely v(k,n) = v(k,k+1)E kv(k+1,n). (20.27) It turns out in fact that Equation (20.27) is the correct condition to prevent arbitrage. Theorem 20.7 If we can find a probability measure Q onΩ such that Equation (20.27) holdsforallnonnegativeintegersk < n≤ N,thenthegivenbondmarketisarbitrage-free. 368 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS Proof. We will show that underQ, eacĥSn satisfies (20.11) and we can then apply Theo- rem 20.4. We know from the definition ofS0 that S0(k+1) = S0(k)v(k,k+1)−1. Moreover, ̂Sn(k+1) = v(k+1,n) S0(k+1) = v(k+1,n)v(k,k+1) S0(k) . Now S0(k) is certainly a k-determined random variable, being a product of k-determined randomvariables,andsothereforeis S0(k)−1.UsingTheorem20.6(b)andinvokingEquation (20.27) Ek(̂Sn(k+1)) = Ek[v(k+1,n)]v(k,k+1) S0(k) = v(k,n) S0(k) = ̂Sn(k) completing the proof. 20.13.4 Short-rate modelling We now deal with a problem that is different from that in previous sections. Instead of being giventheassetprices,andaskedifthemarketisarbitrage-feeornot,wearegivensomeprices and want to determine other prices to satisfy the arbitrage-free condition. For example, refer back to Equation (2.6). This shows in effect that in an deterministic setting, the prices for bonds of one period determine those for all periods. We do in fact have a stochastic version of this formula obtained by taking expectations. Suppose we have a probability measure Q on Ω and random variables v(0,1), v(1,2), …v(N−1,N), where v(k,k+1) is k-determined. We extend this to all our prices by the rule that fork< n, v(k,n) = E k[v(k,k+1)v(k+1,k+2)v(k+2,k+3)…v(n−1,n)]. (20.28) This will indeed result in an arbitrage-free market, since v(k,k+1)Ek[v(k+1,n)] = v(k,k+1)Ek[Ek+1[v(k+1,k+2)…v(n−1,n)]] = v(k,k+1)Ek[v(k+1,k+2)…v(n−1,n)] = Ek[v(k,k+1)v(k+1,k+2)…v(n−1,n)] = v(k,n). (20.29) establishingcondition(20.27).Hereweusedthedefinition(20.28)inthefirstandlastequality, ForthesecondinequalityweusedTheorem20.6(c)notingthatforany k-admissiblesequence 𝜈,t h es e t𝜈 ◦ is a disjoint union of sets𝜈◦ i where each𝜈i is ak+1-admissible sequence – simply add on all possible choices for the last element. So askincreases, the partitioning by k-admissible sequences gets finer. Finally, we use Theorem 20.6(b) for the third inequality. BOND MARKETS 369 Example 20.9 Take N = 3. As remarked above, we need only consider a two-period model. We will take the binomial model, and letQ be the probability measure that assigns equal probability of 1∕4 to each of the four elements inΩ={UU ,UD,DU,DD}. Define the random variablesv(0,1), v(1,2) and v(2,3) by v(0,1) = 0.7, v(1,2)(U) = 0.8 v(1,2)(D) = 0.6.v(2,3)(UU) = 0.9,v(2,3)(UD) = 0.7,v(2,3)(DU) = 0.7,v(2,3)(DD) = 0.5. (a) Use (20.28) to find the distribution underQof the other random variablesv(k,n)f o r k< n, which will make the market arbitrage-free. (b) Show that it is not necessarily true thatv(k,n) = v(k,m)Ekv(m,n)i fm≠ k+1. Solution. (a) The other random variables arev(0,2), v(0,3), v(1,3)which we will calculate directly from Equation (20.28) v(0,2) = E[v(0,1)v(1,2)] = 1 2(0.7×0.8+0.7×0.6) = 0.49. The 2-determined random variablev(0,1)v(1,2)v(2,3) takes the value of 0.7×0.8×0.9 on UU, the value of 0.7×0.6×0.7 on DU, the value of 0.7×0.8×0.7 on UD, and the value of 0.7×0.6×0.5 on DD. Each of these paths has probability 1∕4, so v(0,3) = E[v(0,1)v(1,2) v(2,3)] = 1 4[0.504+0.294+0.392+0.210] = 0.35 Nowv(1,2)v(2,3)takesthevalueof0 .8×0.9onUU andthevalueof0.8 ×0.7onUD so v(1,3)(U) = 1 2(0.72+0.56) = 0.64 Also, v(1,2)v(2,3) takes the value of 0.6×0.7o nDUand the value of 0.6×0.5o nDDso v(1,3)(D) = 1 2(0.42+0.30) = 0.36 (b) v(0,2)E2[v(2,3)] = (0.49)(0.7) = 0.343 ≠ v(0,3) Totiethisexampleinwithpreviousmaterial,thereadermayfinditinstructivetoreproduce the figures shown in Figures 20.7 and 20.8. Figure 20.7 shows all the asset prices for all four assets,Figure 20.8 shows thevalues of̂S k(n)f o rk= 1,2,3. The values of̂S0(n)are of course all equal to 1. Instead of specifying the one-period bond prices, we could equivalently have specified the interest rates as we did in Chapter 2, except that nowik = v(k,k+1)−1 −1, which is a k-determined random variable. We have shown that we can model stochastic interest rates in much the same way as we did in the deterministic model, that is, by choosing the one period rates first. For this reason, this procedure is sometimes referred to as short-rate modelling. What may be surprising at first, is that in the stochastic case the result in far from unique. We can indeed specify any probability measure we want for eachi k, and achieve an arbitrage-free bond market. The procedure doesnottellushowtochoosetherandom variables ik andtheprobabilitymeasure 370 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS 1, 0.7, 0.49, 0.35 0.7‐1, 1, 0.8, 0.64 0.56‐1, 0.8‐1, 1, 0.9 0.56‐1, 0.8‐1, 1, 0.7 0.42‐1, 0.6‐1, 1, 0.7 0.42‐1, 0.6‐1, 1, 0.5 0.7‐1, 1, 0.6, 0.36 Figure 20.7 Example 20.9. Values ofSk(n), k= 0,1,2,3 Q. In practice, this can be motivated by trying to satisfy other conditions that one wants to impose.Forexample,attime0,oneknowsthevalues v(0,n).ButaswesawinExample22.7, thevalues v(0,n)aredeterminedfromtheone-periodrates.Aproblemofsomeinterest,which we will not deal with here, is to choose the distributions of thei k so as to recover specified values ofv(0,n). 20.13.5 Forward prices and rates Wenowexpandonourdefinitionofforwardbondprices,whichweintroducedinSection2.12 Suppose we are given a financial bond market as above. Forj≤ k≤ n,w el e t 0.7, 0.049, 0.35 0.7, 0.56, 0.448 0.7, 0.56, 0.464 0.7, 0.56, 0.372 0.7, 0.42, 0.294 0.7, 0.42, 0.210 0.7, 0.42, 0.252 Figure 20.8 Example 20.9. Values of̂Sk(n), k= 1,2,3 BOND MARKETS 371 ̃vj(k,n) = the forward price to be paid at timej for a 1-unit zero-coupon bond issued at timekand maturing at timen. This will be aj-determined random variable, since it is random depending on the state of nature up to timej. The quantitỹv(k,n) defined in Section 2.12 could be labeled as̃v0(k,n). ThesameargumentasusedinSection2.12showsthattopreventarbitrage,wemusthave ̃vj(k,n) = v(j,n) v(j,k). Indeed,ifthereisanysamplepointatwhichthereisadiscrepancybetweenthesetworandom variables,anindividual couldwaitandseeifthisparticularsamplepointmaterializedattime j, which would be known by the fact that the random variables arej-determined. One could then follow one of two strategies, depending on which way the inequality went, to achieve a sure profit at timen. In particular̃v j(j,n) = v(j,n) which is clear from the definition. In place of the forward prices, we can describe the information available from the bond prices at timejby interest rates as we did before. Forj≤ k, we define aj-determined random variableij(k), theforwardinterestrate for contracts at time timejapplicable to time periodk tok+1, by ̃ij(k) = (̃vj(k,k+1))−1 −1 Note thatik(k) = ik. In the deterministic caseij(k) = ik for allk. It follows immediately from the definitions that v(k,n) = ̃vk(k,k+1)̃vk(k+1,k+2)…̃vk(n−1,n) = [(1+ ̃ik,k)(1+ ̃ik,k+1)…(1+ik,n−1)]−1 (20.30) so that bond prices are determined by the forward interest rates. This suggests an alternative procedure to short-rate modelling for bond prices. The method is to model all the forward interestratesinplaceoftheshortrate i k.Inthiscase,theforwardratesattime0areallknown, sothemodelwillclearlyreproducetheobservedpricesattime0.Onecannot,however,choose thedistributionsoftheforwardratesarbitrarilyasfortheshortrates.Certainconditionsmust be imposed to ensure that the model is arbitrage-free. (See Exercise 20.12). This is another topic that is beyond our scope for further elaboration. 20.13.6 Observations on the continuous time bond market We have concentrated on the discrete time setting for bond market, but many applications involve a continuous time framework. In this section, we give a very quick overview. It is intended mainly to bridge the gap and prepare those who wish to look at other sources and compare the material there with what we have done above. Our market now will consist of zero-coupon bonds maturing at any timet, with a price at times,ofv(s, t).Thiswillbea s-determinedrandomvariableunderanappropriateextensionof thisdefinitiontocovercontinuoustime.Aswellwecandefineanextensionoftheconditional expectationE s. We will not go into details. 372 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS In place of the forward and spot rates of interest, one often wants to deal with the corresponding forces of interest, generalizing the quantity𝛿(t) to a stochastic setting. The forward force of interest, for contracts at timesapplicable to timetis given by ̃𝛿s(t) =− 𝜕 𝜕t logv(s,t), an s-determined random variable. In the deterministic case, when we can assume that our basic Equation (2.1) holds, then log(v(s,t) =l o gv(t)−log v(s) and all forward forces will be equal to𝛿(t) as defined in (8.3). Knowing the forward forces determines bond prices uniquely, since by the definition v(s,t) = e−∫ t s ̃𝛿s(r)dr, which we can view as a stochastic version of Equation (8.4) and as a continuous version of Equation (20.30). We can also define a continuous and stochastic version of the short rate. Let𝛿(t) denote ̃𝛿 t(t). This perhaps requires some care, since the partial derivative applies only to the second variable. Precisely 𝛿(t) = −lim h→0 logv(t,t+h) h , which will be at determined random variable, and one which agrees with our Chapter 8 definition in the deterministic case. As in the discrete time case, this will not uniquely determine bond prices in a stochastic setting. Given any stochastic process for𝛿(t), we can produce an arbitrage-free bond market by taking v(s,t) = E s[e−∫ t s 𝛿(r)dr], under a suitable interpretation of an integral of a random integrand. Notes and references This chapter constitutes a basic introduction to financial markets. For more comprehensive coverage of the basic concepts, see Hull (2014) or McDonald (2012). Readers particularly interestedinthemathematicsofcontinuous-timemodelscanconsultBj ¨ork(2009)orEtheridge (2002) for sources that are not overly advanced. Formanyyears,formula(20.19)wasreferredtoastheBlack–Scholesformula,recognizing the authors of the original paper that was published. Recently, many writers have added the name of Robert Merton, who made important contributions to developing and extending the ideas behind this result. A proof of the extension result on hyperspaces can be found in Steland (2012), Theo- rem 2.4.5. Thosereadingthefinancialliteratureshouldbeawarethattheterminologycandifferfrom the actuarial conventions that we have adopted. Financial economists often use the wordrate of interest to mean the continuously compounded rate, which we have called the force of interest. EXERCISES 373 Exercises 20.1 A one-period financial market has in addition to the bank account, one risky asset. The price at time 0 is 50, and at time 1, the price will either be 52 or 55 each with positiveprobability.Forwhatvalueoftherisk-freeinterestrate rwillthismarketbe arbitrage-free? 20.2 For the option contract introduced in Section 20.5, find an arbitrage opportunity if the option price is 11. 20.3 For the option contract introduced in Section 20.5, find the price ifr is changed to (a) 11%, (b) 9 %. Can you explain why the prices change in this way? 20.4 Show that a trading strategy is self-financing if and only if, for 0≤ n< N, V(n+1)−V(n) = M∑ j=0 𝛼j(n)(Sj(n+1)−Sj(n). 20.5 Assume the data of Example 20.2. Find the self-financing trading strategy for a 2-year American put option with a strike price of 106. Compare the price with the corresponding European put option. 20.6 Consider the financial market with N = 1, M = 1, r= 0, S 1(0) = 100, S1(1) = 130,110, or 80, all with positive probability. (a) Find all possible risk-neutral probability measures. (b) Consider a 1-year call option with a strike price of 100. (This is known as an ‘at-the-money’option).Findtherangeofpossiblepricesforthisoptionthatwill avoid arbitrage. (c) If the price of the option in part (b) is 12, find an arbitrage opportunity. 20.7 Consider the two risky asset market of Figure 20.6 withr= 0; (a) UseTheorems20.4and20.5todecidewhetherornotthismarketis(i)arbitrage- free, (ii) complete. (b) Describe the subspacesLandL 0. 20.8 In Figure 20.3, find the slopes of the two lines representingL0 in terms ofu, d and r. 20.9 WhenpricingoptionswiththeBlack–Scholes–Mertonformula,describehowoption prices change, as changes occurs in volatility, strike price, duration, and risk-free force of interest. Use the formula and put-call parity to compute put and call prices for 3-month European options, on a stock selling now for 100 with a strike price of 110,100and90respectively,assuming 𝜎 is0.2andtheforceofinterestfora1-year periodis0.06.Verifyyourconclusionsabovebyrepeatingthecalculationsforother values of𝜎, 𝛿 and duration. 374 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS 20.10 Foracertainstock,thepriceofacalloptionis7.80andthepriceofthecorresponding put option with the same strike price and expiration date is 1.50. In each of the following scenarios, find the new price of the put option. (a) A change in volatility raises the call price to 11.30. (b) The stock price at time 0 is 100. A 10% increase in the strike price lowers the call price to 5.00. (c) Therisk-freeforceofinterestisdoubledandthetimetoexpirationiscutinhalf, which combine to lower the call price to 7.00. 20.11 Consider a 6-month European call option on a stock now selling for 100, with a strike price of 97. You are given that𝜎 = 0.25 and the force of interest for a 1-year periodis0.10.Youwouldliketoreplicatethisoptionbyatradingstrategyinvolving holdings in the stock and risk-free zero-coupon bonds of face amount 97, maturing in 6 months. You plan to use the Black–Scholes–Merton formula. (a) What is the initial portfolio at time 0? (b) At the end of 2 months, the stock price has risen to 105. What is your portfolio now? (c) At the end of 4 months, the stock price has fallen to 95. What is your portfolio now? 20.12 Give a direct verification for the Black–Scholes–Merton formula in the case that 𝜎 = 0. 20.13 Verify directly from Figure 20.8 that the market of Example 20.9 is arbitrage-free and complete. 20.14 Redo Example 20.9 only assuming that the probability of an upward move is 3∕4 and that of a downward move is 1∕4. 20.15 A bond market has two transitions,U and Dfrom time 0 to time 1. You are given the following forward prices: (i) ̃v(0,1) = ̃v(1,2) = ̃v(2,3) = 0.7; (ii) ̃v 1(1,2) and̃v1(2,3) both take the value 0.8 onUand 0.6 onD. (a) IfQ(U) = Q(D) = 0.5, show that Equation (20.27) is not satisfied. (b) Show that there is no probability measureQ for which Equation (20.27) is satisfied. 20.16 ConsiderthebondmarketwithpricesasgiveninFigure20.7.Consideracalloption onabondmaturingfor1000attime3,withexpirationdatetime2.Findthepriceof the option if the strike price is (a) 900, (b) 650. *20.17 Auniversallifecontractprovidesthatyouraccountwillbecreditedwithaminimum of 4% interest, up to a maximum of 10% interest. Describe the options present in this arrangement. Part IV RISK THEORY

Human: What is the probability of being 1 unit to the west of our starting point at time 3, that is,
atposition −1?

Actuarial AI: 18 An introduction to stochastic processes 18.1 Introduction The purpose of this chapter is to provide background for the remaining chapters in the book. We make much use of the concept of conditioning, so the reader may wish to review Sections A.2 and A.8 of Appendix A. A stochastic process is the tool used to model a quantity varying randomly in time. The following are the essential ingredients. We have an index setT, which gives the points of time that we are interested in. Normally,T will either be the nonnegative integers, 0,1,2,… (discretetime) orthe whole nonnegative line[0,∞) (continuous time).Inboth cases, we will sometimes have a maximum time horizon that we are interested in, in which case,∞ will be replaced by a finiteN). We also need a sample space with a probability measureP, and for each t in T, a random variableX t defined on this space. The random variableXt gives the value at timet, of the random quantity that we are trying to model. Astochasticprocess can then be defined formally as a collection of random variablesXt defined for eachti nas e tT. Wewillillustratebrieflybyconsideringthepriceofacertainstock.Anyonewhohasbeen involved with the stock market can attest that this is indeed a quantity that varies in time, and is subject to all kinds of random influences. Let time be discrete and refer to days. Suppose the stock is selling for 100 per share now, and we know that each day it will either increase by 20%or decrease by 20%. Therefore,X 0 takes a value of 100 with probability 1, whileX1 will take a value of either 80 or 120,X2 will take one of three possible values, 144, 96, or 64, and so on. The details are shown in Figure 18.1. Of course, to complete the model, we have to specify probabilities. We will not do this quite yet, as we want to first discuss the complications that arise. The first principle to observe is that we are not just interested in the distribution of each X t but also in all the possible joint distributions. To illustrate, suppose we want to know the FundamentalsofActuarialMathematics , Third Edition. S. David Promislow. © 2015 John Wiley & Sons, Ltd. Published 2015 by John Wiley & Sons, Ltd. Companion website: http://www.wiley.com/go/actuarial 282 AN INTRODUCTION TO STOCHASTIC PROCESSES 100 120 80 144 96 64 172.20 115.20 76.80 51.20 Figure 18.1 Evolution of stock price probabilitythatthestockwillbepricedat115.20attime3.FromFigure18.1,wecanidentify three mutually exclusive ‘path segments’ leading from 100 at time 0 to 115.20 at time 3. The desired probability can then be written as P(X 0 = 100,X1 = 120,X2 = 144,X3 = 115.20)+P(X0 = 100,X1 = 120,X2 = 96, X3 = 115.20)+P(X0 = 100,X1 = 80,X2 = 96,X3 = 115.20). Many of the questions we ask of stochastic processes are, like the above, concerned with realizations.A realization(sometimescalled samplepath orscenario)ofastochasticprocess is a function defined on the index setT, sendingtto xt, which represents a possible outcome of the process, namely that in which the random variableXt takes the valuext for alltin T. For example, in our example above, the realizationxt = 100×1.2t represents the outcome whereby the stock continually moves upward. It is often useful to view a stochastic process as a model for assigning probabilities to realizations. A technical difficulty arises, however. NormallyT is infinite, andXt takes at leasttwovalues,exceptpossiblyfor X0.Thismeansthereare uncountablymanyrealizations, and so each single realization will normally have probability 0. We must therefore deal with infinite sets of realizations, as these can have positive probability. (This is exactly analogous to the situation with any continuous random variableX where we know that the probability that Xtakes a particular value ofxis always 0, but we are interested in the probability thatX takes values in some infinite set.) Note that in our stock example, an event such as (X 0 = 100,X1 = 120,X2 = 144, X3 = 115.20), which we referred to above, is not by itself a realization (unless our index set T were just {0,1,2,3}). Rather, it is an infinite set of realizations, consisting of all those with the given value ofxt for t= 0,1,2,3, and including all possible values fort> 3. MARKOV CHAINS 283 The reader should realize at this point that if the index setT is in fact finite, then a stochastic process is formally nothing more than a multi-dimensionaljointdistribution. 18.2 Markov chains 18.2.1 Definitions Throughout this section, we will assume that our index setT is the nonnegative integers and that eachXt is discrete. Toanswerrelevantquestionsaboutastochasticprocess,wewanttobeabletocomputeall finite joint distributions. Normally, however, we do not do so directly but deduce these joint distributionsthroughsomeotherinformationthatwecandetermineaboutourmodel.Suppose we are at timek, and we want to predict what will happen to our quantity at timek+1. This canbequitecomplicatedasitcoulddependontheentirepasthistoryofwhathappened upto timek.Inmanyapplications,thisissimplified,sincetheonlyrelevantpartofthispasthistory forthispredictionistheactualvalueattime k,andwegetnofurtherinformationfromlooking at the values before that time. The following formulates this precisely. Definition 18.1 A discrete-time stochastic process with discrete random variables is called a Markov chain, if given any finite sequencex 0,x1,x2,…,xk+1, wherexi is a possible value of Xi, P(Xk+1 = xk+1|Xk = xk,Xk−1 = xk−1,…,X0 = x0) = P(Xk+1 = xk+1|Xk = xk). (18.1) To illustrate, consider the stock example given above. Suppose that we decide that each daythestockwillmoveupwithprobabilityof2 ∕3andmovedownwithprobability1∕3.This isclearlyaMarkovchainsinceboththelefthandsideandrighthandsideof(18.1)are2 ∕3if xk+1 = 1.2xk,1 ∕3i fxk+1 = 0.8xk, and 0 in all other cases. Suppose, however, that we decide that these probabilities will hold only when the stock has made two different movements on the previous two days. On the other hand, we decide that if the stock moves up two days in a row, it signifies a trend, and the probability of an upward move on the next day changes to 3∕4, while if the the stock moves down two days in a row, it signifies a pessimistic attitude, and the probability of an upward move on the next day is only 3∕5. This would no longer be a Markov chain, since the probability ofX k+1 is clearly influenced by the values ofXk−2 and Xk−1 as well as that ofXk. For example, takingk= 3, P[X4 = 92.16|X3 = 115.20,X2 = 96,X1 = 80] = 1 4, but P[X4 = 92.16|X3 = 115.20,X2 = 96,X1 = 120] = 1 3. In a Markov chain, these would both have to equal the same number, namelyP[X4 = 92.16|X3 = 115.20]. The important feature of the Markov property is that it allows us to compute all relevant probabilities once we know the probabilities of each ‘branch’ of the tree-like structure, as we have drawn in Figure 18.1. To elaborate, we introduce some fundamental notation. 284 AN INTRODUCTION TO STOCHASTIC PROCESSES For any integersk≤ n,l e t pxy(k,n) = P(Xn = y|Xk = x). (18.2) The probability of any branch is of the formpxy(k,k+1) and we can multiply the proba- bilities of each branch to get the probability of any path. Finally, we can compute the general probability p xy(k,k+n) by adding the probabilities on all paths that lead from a value ofxat timekto a value ofyat timen. Considerourstockexamplewiththe(2 ∕3,1∕3)probabilitiesoftherespectiveupordown move. We can now easily answer our question, and find the probability that the stock price is 115.20 at time 3, that isp 100,115.20(0,3) There are 3 paths leading from the starting value of 100 to the value of 115.20 at time 3. Each of them consists of two up moves and one downmove,andwillhaveprobabilityequalto 2 3 × 2 3 × 1 3 = 4 27,andsothetotalprobabilityin question is 12∕27. Inmanycommonapplications(asinthisone),thereisthefurthersimplifyingfeaturethat px,y(k,k+1) is independent ofk, and can be denoted by justpxy. In this case, we say that we havestationarytransitionprobabilities.(Thisfeatureisalsoreferredtoas timehomogeneity.) In connection with stochastic processes, the wordstationary can be thought of as referring to a process ‘without a watch’. In the present context, it means that whenever the evolving quantity takes a value ofx, the probability that it takes a value ofyat the next stage is always the same, regardless of the particular time. 18.2.2 Examples WenowlookatsomeotherexamplesofMarkovchains.Oneofthemostfamousisthe random walk. An indecisive person goes for a walk but cannot decide whether to go east or west. A coin is flipped and the person goes 1 unit east if a head comes up or 1 unit west if a tail comesup.Aftereachmove,thecoinisflippedagainandtheprocedurerepeated.Supposethe probability of a head coming up isp. LettingX k refer to his position east of the starting point at timek, we have the process shown in Figure 18.2. Thisissimilartothestockquestionaskedabove.Wehavethreepossiblepath segmentsleadingto −1attime3,eachwithprobability p(1−p)2,sotheansweris3p(1 −p)2. Thissamemodelappliestomanysituations.Imagineagamblerrepeatedlyplayingagame withanevenmoneypayoff,suchasbettingonblackatroulette.Inanysingleplay,thisperson eitherwins1unitwithprobability porloses1unitwithprobability1 −p.IfweletX k denote the total winnings afterk plays (a negative amount signifying a loss), then we have exactly the random walk process as described above. Many of the processes that we will deal with can be viewed in this gambling context. It will be convenient to make a slight adjustment. Instead of keeping track of the amount won or lost, we will keep track of the total fortune of the gambler starting from an initial fortune of u. This is just a matter of addinguto each entry. For example, if the gambler starts play with a fortune of 10 units, then the diagram of the process would be as shown in Figure 18.3. Considerasituationwherethewagerateachstageismorecomplicatedthanasimpleeven moneybet.Supposethatthereturntothegambleronthisbetissomediscreterandomvariable G. (In the case above,Gsimply took the value 1 with probabilitypand −1 with probability 1−p.)Thegambler startswithafortuneof u,andwewanttoconsiderthestochasticprocess MARKOV CHAINS 285 3 1 1 –1 0 2 –1 –3 –2 0 Figure 18.2 Random walk 1010 11 13 8 9 12 11 9 7 Figure 18.3 Gambler’s fortune 286 AN INTRODUCTION TO STOCHASTIC PROCESSES Un which equals the fortune of the gambler at timen. LettingGk be the return at timek,w e can write Un = u+G1 +G2 +⋯+Gn, (18.3) where theGis are independent and each distributed asG. This is clearly a Markov chain with transition probabilities given by px,y(k,k+1) = P(G= y−x), and since the right-hand side is independent ofk, we see also that this process has stationary transition probabilities. Note that it would still be a Markov chain, but not necessarily a stationary one, if the gambler were to change the bet at various times, as long as this was done independently of thepasthistoryofwinnings.Anexampleofthiswouldbeforthegamblertodecidethatafter fiveturnsattheroulettetable,regardlessofwhathappens,hewillswitchtoblackjack.Inthis case,thedistributionsofthevarious G n arenotallthesame.If,however,thegamblerdecided that he would play roulette until he had five consecutive losses and then switch to blackjack, the resulting process wouldnothave the Markov property. We return to the process given by (18.3) in Chapter 23 where it will play an important role in modelling the surplus of an insurer. 18.3 Martingales Inthissection,webrieflyintroduceanotherimportantclassofstochasticprocesses,whichwe will make much use of later. Definition 18.2 A discrete-time stochastic process is amartingaleif E(X k+1|Xk = xk,Xk−1 = xk−1,…,X0 = x0) = xk, forall kandvalues x0,x1,…,xk.Inotherwords,atanytime,theexpectedvalueofourquantity at the next time period is exactly what it is now. Note that in the case of a Markov process, the requirement above simplifies to E(Xk+1|Xk = xk) = xk. Is our stock price process with the 2∕3 and 1∕3 probabilities a martingale? The answer is no, since E(Xk+1|Xk = x) = 2 3 ×1.2x+ 1 3 ×0.8x= 3.2x 3 > x. A process like this, for which the expectation of the future random variable is always greater than or equal to the present one, is known as asubmartingale. It would only be a martingaleifthe2∕3werereplacedby1∕2.Indeed,wewouldnotexpectastochasticprocess for stock prices to be a martingale, since it would mean that there was no tendency for the pricetoincrease.However,thereasonforpurchasingstockinthefirstplace,giventheinherent risk, is the expectation of capital gains as the stock increases in value. FINITE-STATE MARKOV CHAINS 287 What about the process given by (18.3)? We have E(Uk+1|Uk = x) = E(Uk +Gk+1|Uk = x) = x+E(G). This process will be a martingale if and only ifE(G) = 0, that is, if and only if the bet made by the gambler is a fair bet. The martingale concept was in fact introduced originally as the model of a fair game. In the usual casino games,E(G) < 0, and we would not have a martingale. Suchaprocess, wheretheexpected valueofthefuturerandom variable isalways lessthanorequaltothepresentone,isknownasa supermartingale.Amartingaleistherefore both a supermartingale and a submartingale. (To remember the terminology, note that the modifier applies to thecurrentvalue. So asubmartingale means that at any time, the current value of the process isunderthe expectation of the future value.) 18.4 Finite-state Markov chains Sometimes, it is convenient to relabel with integers the possible values that can be taken by therandom variablesinaMarkovchain.Werefertotheseintegersasthe statesofthesystem and say that the system is in statei at timek if X k takes the valuei. In fact we often just take N, the number of the state, as the underlying random variable in place. ofX, so that our fundamental probabilitiespij(k,n) will be the probability of being in statej at timen given that the process was in statei at timek. In this section we consider Markov chains with a finite number of states and develop matrix methods for investigating their properties. In this chapter we confine our attention to the stationary case. Nonstationary chains are needed in Chapter 19 and will be introduced at the beginning of that chapter. Starredsectionscontainsomewhatmoreadvancedmaterialandthesecanbeomittedatfirst reading (although we do refer back to one of the results in one of the Chapter 23 examples). 18.4.1 The transition matrix Suppose we have a stationary Markov chain withNstates. We will number them {0,…,N− 1}. (Some authors use the numbers 1 to N instead). By the stationary condition, the quantity p ij(k,k+1) the probability of moving from stateito statejin one step, is, independent ofk, and we can write it as justpij. The matrixP, with entries ofpij in theith row andjcolumn is known as thetransitionmatrix of the chain. Note that the only condition required for anN×N matrix to be the transition matrix of some Markov chain is that all entries are nonnegative, and each row sums to 1. Remark The reader should keep in mind that we have started the indexing at 0, so the top row (or left column) of the matrix will be considered as row (or column) number 0,not number 1. Example 18.1 A box contains two balls, either of which can be red or yellow. At each stage,aballischosenatrandomandreplacedwithaballoftheoppositecolour.Let X n bethe number of red balls in the box at timen. Find the transition matrix. Solution. Herewehaveathree-stateMarkovchain.Wewillnumberthestatesbythenumber ofredballsinthebox.Ifthereare0or2redballs,wearesuretomovetostate1,whileifthere 288 AN INTRODUCTION TO STOCHASTIC PROCESSES is 1 red ball, we move to either state 0 or 2, with equal probability. Therefore the transition matrix is P = ⎛ ⎜ ⎜ ⎝ 010 1∕201 ∕2 010 ⎞ ⎟ ⎟ ⎠ . 18.4.2 Multi-period transitions We now wish to calculate the general probabilitiespij(k,k+m) which by stationarity will be equal topij(0,m). We start by calculating this form= 2. Suppose we move from statei to statekattime1andthenfromstate ktostate jtime2.Weknowfromourpreviousdiscussion onMarkovchainsthattheprobabilityofthistwo-stepmoveoccurringisjust pikpkj.Summing this over all statesk, we get the probability of moving fromitojafter two periods as N∑ k=1 pikpkj, (18.4) which, by the ordinary rules of matrix multiplication, is justP2(i,j)t h e(i,j)th entry of P×P = P2. The same argument can be repeated to show the important fact that in a stationary chain, pij(0,m) is equal to the entry in rowiand columnjofPm (18.5) (Notethat P0 isjusttheidentitymatrix,usuallydenotedby I,withentriesof1onthemain diagonal, and entries of 0 elsewhere.) 18.4.3 Distributions Given a finite-state Markov chain, what is the distribution ofXn? This is a basic question that we ask about any stochastic process. The distribution can be given as anN-dimensional vector𝜋n, whoseith entry, denoted by𝜋n(i), equalsP(Xn = i). This will depend of course on 𝜋0,thevectorgivingtheinitialdistributionattime0.(Incertainapplications,wemightknow the value ofX0, in which case,𝜋0 will simply be a vector with a single entry of 1, and other entries equal to 0.) Starting withn= 1, we calculate P(X1 = j) = ∑ i 𝜋0(i)pij, which is just thejth entry of the vector obtained by multiplying the vector𝜋0 (viewed as a 1×N matrix) on the right byP. The same argument holds for anyn, and we can conclude 𝜋n = 𝜋0Pn. (18.6) Example18.2 InExample18.1,supposewestartwithauniformdistribution,thatis,there is a 1∕3 chance thatX0 takes each of the values 0,1,2. What is the probability thatX101 = 1? FINITE-STATE MARKOV CHAINS 289 Solution. Computing large powers of matrices is often done by calculating eigenvalues, but we can avoid that here. After a couple of multiplications, we see thatP3 = P. Therefore P5 = P3P2 = PP2 = P, and similarly,anyoddpower ofP equalsP. From (18.6), 𝜋101 = (1∕3,1∕3,1∕3) ⎛ ⎜ ⎜ ⎝ 010 1∕201 ∕2 010 ⎞ ⎟ ⎟ ⎠ = (1∕6,2∕3,1∕6). The probability thatX 101 equals 1 is 2∕3. Remark Somewritersdefinethetransitionmatrix Pbytakingtheentryinrow iandcolumn jas the probability of moving from statejto statei. In this case, columns, rather than rows, add to 1. The vectors𝜋n are written as column vectors and in (18.6), we multiply𝜋0 on the leftto get𝜋n. ∗18.4.4 Limiting distributions The distributions𝜋n will normally continue to change with time, but in many applications, we would like to show that there is some limiting distribution𝜋 which is independent of the initial state at time 0. That is, for any initial statej, and all statesi, the resulting probabilities 𝜋n(i)willconvergeto 𝜋(i)as napproaches∞.Thiswouldenableustopredictwithreasonable accuracytheprobabilitiesofbeinginvariousstates,providedtheprocesshasbeencontinuing forasufficientlylongtime.Weconsiderheretheproblemoffindingthislimitingdistribution, provided it exists. The last provision is necessary, as indicated by the following examples where a limiting distribution does not exist. Consider the chain of Example 18.1. If the process is in state 0 or 2, it will move to state 1 in one transition. If it is in state 1, it will move to state 0 or 2 in one transition. So if we start, say in state 0, the process is sure to be in state 0 or 2 at even times, and in state 1 at odd times. Clearly, no limiting distribution can exist. This is an example of what is known as a periodicchainofperiod2.Ingeneral,thismeanswecandividetheset Sofallstatesintotwo disjoint subsets,S 1 and S2, such that in any one transition, all states inS1 move toS2 and all states inS2 move toS1. More generally, we could have chains of perioddwhere we can find d pairwise disjoint subsets, such that from each subset, we cycle through the other sets in a fixed order, and return to the original set afterdtransitions. Another example where a limiting distribution will not exist is the chain with transition matrix P = ⎛ ⎜ ⎜ ⎝ 1∕21 ∕20 1∕21 ∕20 00 1 ⎞ ⎟ ⎟ ⎠ . In this case, if we start in state 2 at time 0, then we remain in state 2 forever. If we start in state 0 or 1, we will reach a limiting distribution, which is to be in state 0 or 1 with equal probabilities.(In fact, this limiting distribution is achieved exactly at time 1). The problem hereisthatthelimitingdistributionsvarysaccordingtotheinitialstate.Thisisanexampleof what is known as areduciblechain. This means we can divide the setSof all states into two nonempty disjoint subsetsS 1 and S2 such that any state inS1 transfers to another state inS1 290 AN INTRODUCTION TO STOCHASTIC PROCESSES after one transition and any state inS2 transfers to another state inS2 after one transition. We can really consider such a Markov chain as two separate chains, one comprising the states in S 1 and the other comprising the states inS2. A simple condition that ensures the existence of a limiting distribution is that there is a positive integern such that all entries ofPn are positive. That is, given any statesi,j (not necessarilydistinct),thereissomechanceofgettingfrom itojinntransitions.Aproofofthis result is beyond the scope of this book. It is, however, not difficult to see that this condition rules out both periodic and reducible chains. See Exercise 19.12. If we know that a limiting distribution𝜋 exists, then there is a straightforward procedure for finding it from the matrixP. We know that𝜋 = lim n→∞ 𝜋n (in the sense of converging at each i as described above). By continuity considerations, we must have𝜋P = limn→∞ 𝜋nP. From (18.6), we know that𝜋nP = 𝜋0PnP = 𝜋0Pn+1 = 𝜋n+1. But clearly𝜋n+1 also converges to𝜋. This establishes that 𝜋 = 𝜋P. (18.7) Note that a solution𝜋 of Equation (18.7) need not be a limiting distribution, since none may exist. It is true that once we have a distribution𝜋 satisfying this equation, we will remain with that distribution forever, but it is possible that from some initial states we will never converge to𝜋. Example 18.1 is an illustration. We have exactly one solution to (18.7), namely 𝜋 = (1∕4,1∕2,1∕4), but as we have shown, we will not approach this for all initial distributions. (It is of interest to note that this is the distribution we would get if we started the process by choosing the colour of each ball randomly.) ∗18.4.5 Recurrent and transient states The states of a stationary Markov chain can be divided into two classes. Given any statej,l e t fj be the probability that, starting in statej, the process will return to that state. Definition 18.3 Statejissaidtobe transientiffj < 1.Inthiscase, thereissomechance that the process will never return. Statejis said to berecurrentif fj = 1. In this case, the process is sure to return. Example 18.3 Classify the states of the Markov chain with transition matrix P = ⎛ ⎜ ⎜ ⎝ 3∕41 ∕40 1∕21 ∕20 1∕403 ∕4 ⎞ ⎟ ⎟ ⎠ . Solution. State2istransient,sincestartingatstate2,thereisapositive chance ofmoving to state 0. From there, only states 0 and 1 can be reached, and there is no return to state 2. Wewillshowthatstates0and1arerecurrent.Supposeweareinstate0.Wefirstobserve thatitiscertainthatwewilleventuallygettostate1.Foranypositiveinteger n,theprobability of staying in state 0 forever is certainly less than the probability of staying in state 0 for the nextntransitions, which is (3∕4) n. As this quantity approaches 0, we see that the probability ofstayinginstate0forever is0,andsowithprobability1wewilleventually move tostate1. Similarly,ifweareinstate1,wearecertaintomovetostate0.Therefore,ifweareinstate0, we are certain to move to state 1 and then back to state 0 Arguing in the same way for state 1, we see that both states are recurrent. FINITE-STATE MARKOV CHAINS 291 This is a fairly simple case, and in general it might not be so easy to classify the states from the matrix. We can, however, adapt the argument in Example 18.3 to prove a general result, which can be of great help in the classification. Definition 18.4 We say a statejis reachablefrom a stateiif there is a positive probability ofeventuallymovingfromstate itoj.Intermsofthematrix P,thisconditioncanbestatedas P n(i,j) > 0f o rs o m en. Theorem 18.1 Ifiisrecurrentandjisreachablefromi,thenjisrecurrent. Proof. Starting in statej, we are certain to eventually reach statei, for if not, then there would be a positive probability of going fromitojand never returning toi, contradicting the fact thatiis recurrent. Now, starting in statei,l e t𝛼be the probability that we will return to state iwithout ever hitting statej. Sincejis reachable fromi, we must have that𝛼< 1. The probability that starting in stateiwe will makenreturn visits toiwithout ever hitting state jis 𝛼n, which approaches 0 asngoes to∞. Therefore, the probability is 0 that, starting ini, we never reach statej, which means that we are in factcertainto eventually reach statej.T o conclude, starting in statej, we are certain to reach stateiand certain to come back to statej from statei, showing that statejis recurrent. Starting in a recurrent state, we must always remain in recurrent states. What happens if we start in a transient state? Theorem 18.2 In a finite-state stationary Markov chain, there is at least one recurrent state.Moreover,startingfromanytransientstate,wemusteventuallyreacharecurrentstate. Proof. Given a transient statej, consider all realizations of the process for which there is at leastoneoccurrenceof j.Theprobabilitythattherewillbeexactlyonesuchoccurrence,given that there is at least one, is 1−f j, the probability of never returning toj. The probability that therewillbeexactlytwosuchoccurrences,giventhatthereisatleastone,isjust fj(1−fj).The process must return once, and then never return again. Continuing, we see that the number of occurrences ofj, less 1, given that there is at least 1, has a geometric distribution.(See Section A.11.3) Now a geometric distribution is a proper frequency distribution with no probabilityofassumingthevalue ∞.Ourconclusionisthattheprobabilityofinfinitelymany occurrences of any transient statej is 0. Consider the event that the chain never visits a recurrent state. If this occurs, since there are only finitely many transient states, one of them must appear infinitely often, but as we have seen the probability of this is 0. This means that with certainty, we must reach a recurrent state. WecanapplythelasttwotheoremstoExample18.3.Onceweseethatstate3istransient, weknowbyTheorem18.2thateitherstate1orstate2isrecurrent,andthenbyTheorem18.1 (or even by symmetry) they both must be recurrent. Note that Theorem 18.2 need not hold for an infinite-state chain. For a trivial example, taketherandomwalkwheretheprobabilityofmovingtotherightis1.Allstatesaretransient. Some transient states are ‘less transitive’ than others. That is, the expected time spent in that state is longer. Indeed, for many applications we may be interested in this expected time spent in a certain transitive statej, that is, the expected number ofnfor whichX n = j. (Thereisnopointinaskingthisquestionforarecurrentstate,since,bydefinition,theprocess 292 AN INTRODUCTION TO STOCHASTIC PROCESSES is in the state for infinitely many values ofn.) Of course, this expectation may depend on the starting statei. Once again this question is of interest only for a transitive starting state i, since for a recurrent starting state, we know by Theorem 18.1 that the answer is 0. We calculate this expectation by the familiar trick of looking atindicatorrandom variables (see the end of Section A.5). Fix such a starting stateiand consider the random variableI n that takes the value 1 ifXn = j,o r0i fX n ≠ j. Then the expected number of visits to statej is just E[∑∞ n=0 In] = ∑∞ n=0 E(In), since A.22 extends to infinite sums for nonnegative random variables. SinceE(In) = pij(0,n), we have Expected visits to statejstarting from statei= ∞∑ n=0 pij(0,n). (18.8) The right hand side of (18.8) may seem quite formidable to calculate. It is possible to use eigenvalues of the transition matrixPto shorten this, (see Section 19.3.4) but there is a much simpler method, based on an idea that is useful in many contexts. Recall the formula for an infinite geometric progression. For a numberx, of absolute value less than 1, (1−x) −1 = 1+x+x2 +⋯. Similarly, it can be shown that for a matrixQ with sufficiently small entries, the matrix I−Qis invertible, and (I−Q)−1 = I+Q+Q2 +⋯. Supposethatthetransientstatesinourmatrixarenumbered0, ….m−1andtake Qtobe them×msubmatrix consisting of the firstmrows and firstmcolumns. Using Theorem 18.1, we can show that fori,jbetween 0 andm−1, the (i,j)th entry ofQn is equal topij(0,n) with the final result that for any two transient statesi,jnot necessarily distinct, Expected visits to statejstarting from statei= the (i,j)th entry of (I−Q)−1 (18.9) Example18.4(Randomwalkwithabsorbingbarriers) Considerarandomwalkonfour consecutive points 1,2,3,4 on a line. Starting at either 2 or 3, the process moves right with probability 2∕3 or left with probability 1∕3. Whenever the process gets to either 1 or 4, it remains there forever. Classify the four states as to recurrent or transitive, and for each pair (i,j)oftransitivestatesfindtheexpectednumberoftimestheprocesswillbein jstartingfrom statei. Solution. Points 2 and 3 are clearly transient. Points 1 and 4 are clearly recurrent, in fact a particular type of recurrent state, known as anabsorbingstate, which means that once there, you never leave. Numbering point 2 as state 0 and point 3 as state 1, for the transient state submatrix Q,w eh a v e Q = ( 02 ∕3 1∕30 ) , I−Q = ( 1 −2∕3 −1∕31 ) ,( I−Q) −1 = ( 9∕76 ∕7 3∕79 ∕7 ) . The final matrix gives the expected number of visits. For example, starting at point 2, the expected number of times the process visits point 3 is 6∕7. POISSON PROCESSES 293 18.5 Introduction to continuous time processes We now turn our attention to processes where the index setT takes all values in the interval [0,∞). We will also allow continuous rather than discrete random variables. One goal is to define continuous-time analogue for the process given in the discrete case by (18.3). In Chapter 23, we apply this to modelling the surplus of an insurer. Another goal is to introduceBrownian motionwhich can be viewed as a type of continuous-time random walk, which we apply in Chapter 20. Notation For continuous-time processes, we will write the time variable in brackets rather than as a subscript. That is, we writeX(t) in place ofX t. We writeX ∼ Y where Xand Y are any two random variables to mean that they have the same distribution. We begin with a few general definitions that capture some of the features we introduced in the discrete time setting. Given a stochastic processX(t) and two timess< t, the random variable X(t)−X(s)i sc a l l e da nincrementof the process, since it gives the increase in the value over the period running from timesto timet. Definition 18.5 We say that the process hasindependentincrements if the increments over disjoint time intervals are independent. This constitutes a strong version of the Markov property. Given timess< t, we can write X(t) = X(s)+X(t)−X(s).Thisshowsthatthevalueof X(t)cancertainlydependonthevalue of X(s), but we get no additional information from looking at times befores, since both X(t)−X(s) andX(s) are independent of what happened in the interval [0,s). Definition 18.6 We say that the process hasstationaryincrements if the distribution of any incrementdependsonlyonthelengthofthetimeintervalandnottheparticularstartingpoint. That is, given anys,t,h> 0,we require that X( s+h)−X(s) ∼ X(t+h)−X(t). This constitutes a strong version of the assumption of stationary transition probabilities that we made for discrete Markov chains. 18.6 Poisson processes Suppose we have a particular ‘event’ that we are interested in, occurring repeatedly and randomly in time. This could be an insurance claim or the arrival of a person at a queue, or a number of other possibilities. Acountingprocess is a stochastic processN(t) that counts the numberofsuch‘events’thathaveoccurreduptotime t.Formally,itisjustanycontinuous-time process that takes nonnegative integer values, and such that all realizations are increasing. A particularrealizationcanthenalwaysbedrawnasanincreasingstepfunction.Wewillalways assume that in any counting process,N(0) = 0. In other words, the counting starts at time 0 294 AN INTRODUCTION TO STOCHASTIC PROCESSES before any events have occurred. A major application in this text will be to insurance claims, which is discussed in Chapter 23. We will confine attention to the particular case of Poisson processes defined as follows. Definition 18.7 A counting processN(t)i sc a l l e daPoisson processwith rate𝜆 if it has stationary and independent increments and if, for allh> 0, N(h) ∼ Poisson(𝜆h). It follows from the stationary increment assumption that, given anyt> 0, the random variableN(t+h)−N(t) ∼ Poisson(𝜆h).Inotherwords,aPoissonprocessissimplyacounting processwithindependentincrements,suchthatthenumberofoccurrencesinanytimeinterval is a Poisson distribution, with parameter proportional to the length of the interval. WhenshouldwechooseaPoissonprocesstomodelacountingsituation?Thereisanother characterization of Poisson processes that gives some insight into answering this question. Suppose we assume that the increments are indeed stationary and independent. The alternate formulation says essentially that we will get a Poisson process if it is ‘highly unlikely’ to have more than one event occurring in a ‘sufficiently small’ time interval. Therefore, if you feel that this is the case for the particular event you are trying to model, you can be justified in choosing the Poisson process. To derive this characterization, we must first give precise meaning to the phrases ‘highly unlikely’ and ‘sufficiently small’. This is done conveniently through the ‘littleo’ notation, which we will now review. We say that a functionf defined on an interval [0,b]i so(h)i f lim h→0 f(h)∕h= 0. This means thatf is getting small rapidly ashgets small, more rapidly thanhitself. For example, f(h) = h2 iso(h),while f(h) = √ hisnot.Weoftenwritethesymbol o(h)forsuchafunction. For example, we would write e𝛽h = 1+𝛽h+o(h), as can be seen from the Taylor series expansion. It is clear that, given two functionsf and g that are botho(h), their sumf +g is o(h), and for any constantc the functioncf is o(h)a s well. We can now state the desired characterization. Theorem 18.3 A counting process that has stationary and independent increments is a Poissonprocesswithrate 𝜆ifandonlyifthefollowinghold: (i) P(N(h) = 1) = 𝜆h+o(h), (ii) P(N(h) ≥ 2) = o(h). Partial proof. One direction is clear. For a Poisson process with rate 𝜆,w eh a v e P[N(h) = 1] = 𝜆he −𝜆h = 𝜆h[1−𝜆h+o(h)] = 𝜆h+o(h). Similarly, we haveP[N(h) = 0] = e−𝜆h = 1−𝜆h+o(h). Therefore,P[N(h) ≥ 2] = 1−P[N(h) = 0]−P[N(h) = 1] = 1−(1− 𝜆h+o(h))−(𝜆h+o(h)) = o(h). The converse is the more difficult part. We assume conditions (i) and (ii) and must show thatN(h)isPoisson.Therearevariousmethodsofdoingthisandwewillnotelaboratefurther. A nice proof can be found in Ross (2010, Theorem 5.1) where this is done by calculating the moment generating function ofN(h). BROWNIAN MOTION 295 18.6.1 Waiting times Givenanycountingprocess, N(t),thereisanassociated waiting-timeprocess,Wn,n= 1,2, …, where Wn is the time between the (n−1)th event and thenth event. We take the 0th event as occurringattime0.So,forexample,ifthefirsteventoccursattime1,thesecondattime1.7, the third at time 2.3, we would haveW 1 = 1, W2 = 0.7, andW3 = 0.6. For some problems, it is more convenient to deal withWn rather thanN(t). Note that the waiting-time process is a discrete-time process (although the index does not refer to times exactly) with continuous random variables, while the counting process is a continuous-time process with discrete random variables. A natural question is to investigate the waiting-time process for a Poisson process, and this is easily answered. Let𝜆 be the rate of the process. Suppose thatW 1 = w1,W2 = w2,…,Wn−1 = wn−1. What is the distribution ofWn?I fs= w1 +w2 +wn−1,t h e(n−1)th occurrence was at times, and in order forWn to be greater than or equal tow, we require that there be no occurrences in the interval (s,s+w], which, by the stationary increment assumption,hasthesameprobabilityasnooccurrencesintheinterval(0, w].Thisprobability isjuste −𝜆w,whichiseasilyrecognizedasthesurvivalfunctionofanexponentialdistribution. We conclude that theWn are independent and each is distributed as Exp(𝜆). Another random variable that is of interest in many applications isTn, the time of thenth occurrence(sometimescalledthe ntharrivaltime).Clearly, Tn = W1 +W2 +⋯+Wn,andit is easy to see using moment generating functions thatTn is distributed as Gamma(n,𝜆). 18.6.2 Nonhomogeneous Poisson processes In many counting processes, the stationarity assumption is not realistic, as we can expect the rateofoccurrence tovarywithtime.Tomodel this,weuseatypeofprocessthatissimilarto a Poisson process, except the rate𝜆 is no longer a constant but rather a function oft. Definition 18.8 A counting process is called anonhomogeneous Poisson processwith intensity function𝜆(t) if it has independent increments and, for allt> 0, (i) P(N(t+h)−N(t) = 1) = 𝜆(t)h+o(h), (ii) P(N(t+h)−N(t) ≥ 2) = o(h). It can then be shown fors< t, the incrementN(t)−N(s) ∼ Poisson(𝜙(s,t)), where 𝜙(s,t) = ∫ t s 𝜆(r)dr. We omit the proof. Note that when𝜆(t) is a constant𝜆, then𝜙(s,t) is just equal to (t−s)𝜆, and we have exactly the same conclusion that we had before in the case of a regular Poisson process. 18.7 Brownian motion 18.7.1 The main definition We now look at another continuous-time stochastic process which has several applications. This is intended as an introduction and we will not supply all proofs. 296 AN INTRODUCTION TO STOCHASTIC PROCESSES A stochastic processX(t),0 ≤ t< ∞, is called aBrownianmotion process with variance parameter 𝜎2,f o rs o m e𝜎>0, if it satisfies the following four conditions: (i) X(0) = 0; (ii) the process has independent and stationary increments; (iii) for eacht> 0,X(t) is normally distributed with mean 0 and variance𝜎2t; (iv) the realizations,t→ xt are continuous functions oft. NotethatonaformalbasisthedefinitionissimilartothatofaPoissonprocess,althoughthe nature of the processes are quite different. Both have independent and stationary increments. In the Poisson case, the increments have a Poisson distribution with expectation proportional tothelengthoftheinterval,andintheBrownianmotioncase,theyhaveanormaldistribution with variance proportional to the length of the interval. In the Poisson case, the realizations are step functions which involve sudden jumps, in contrast to the continuous realizations of Brownian motions. AstandardBrownianmotionisoneinwhich 𝜎 2 = 1.Wewilldenotethisby B(t).Forthe Brownian motionX(t) with variance parameter𝜎2 it is clear that X(t) = 𝜎B(t). 18.7.2 Connection with random walks A Brownian motion can be viewed as acontinuousrandom walk, where moves are made at each instant of time. Consider first our simple random walk of section 18.2.2. We start at 0 on the real line, and each time unit, we move one unit either to the right or left, with equal probability.Letuscomputethedistributionof X(t),ourpositionattime t.Itiseasytoseethat this is related to a binomial distribution. Precisely X(t) ∼ 2Bin(t,1∕2)−t, (18.10) from which it is clear that (see (A.41)) E(X(t) = 2t(1∕2)−t= 0, Var(X (t) = 4t(1∕4) = t. We see that our simple random walk has the mean and variance of condition (iii) in the definition of a standard Brownian motion. Let us now speed up our random walk by making moves more frequently. Instead of movingeachtimeunit,letusmoveevery1 ∕mofatimeunitwhere missomepositiveinteger. Of course, we also want to change the length of each move. One may think that instead of moving1unitateachstep,weshouldnowmove1∕m ofaunit.However,thiswillnotpreserve the variance condition that we want. To do so, we need to make much larger moves. A key fact is that we will need to make the length of each move equal to 1 √ m. So for example, if m= 4,wewouldmakemovesoflength1∕2attimes1∕4,2∕4,3∕4,etc,whileif m= 100,we would make move of length 1∕10 at time 1∕100, 2∕100 etc. LetX(m)(t) denote our position at timet under this new arrangement. It is not hard to see that this will be the same as the BROWNIAN MOTION 297 position at timemtof the simple case with moves of 1 unit each time period, only multiplied by the length of each move, which is 1∕ √ m. From (18.10) X(m)(t) ∼ 2√ m Bin(mt,1∕2)− mt√ m , and we see as above that E(X(m)(t) = 0,Var(X(m)(t) = t. The standard Brownian motion B(t) can be viewed as a limiting case of X(m)(t)a s m approaches ∞. ∗18.7.3 Hitting times Formanystochasticprocessesweareinterestedinthetimethatitfirstreachesacertainpoint. These random times are known ashittingtimes. We will deduce hitting time distributions for Brownian motion, by invoking one of its key properties, known as thesymmetry principle. This simply says that if the process is at a pointb at time s, then at any later timet the distribution must be symmetric aboutb. In other words, given thatB(s) = b, then for any numbersh< k. P(b+h≤ B(t) ≤ b+k) = P(b−k≤ B(t) ≤ b−h) (18.11) This feature is clear for the simple Random walk, where we are just as likely to move to the right as to the left, and therefore is true for Brownian motion by viewing it as a limiting case of random walks. Let B(t) be a standard Brownian motion, and for any pointa,w el e t T a =i n f{t: T(t) = a} To compute the distribution ofTa, we will derive a relationship between the distribution of Ta and distribution ofB(t). Suppose first thata> 0. Then by Equation (A.30) P[B(t) ≥ a] = P[B(t) ≥ a|Ta ≤ t]P[Ta ≤ t] +P[B(t) ≥ a|(Ta > t]P[Ta > t]. We note first that the second term on the right equals 0. Indeed, IfTa > t then the first time the process hitsais after timet, and the value at timetcould not be greater than or equal to a, since if so, by continuity of the realizations, we would have needed to reachasometime before timeT. Consider now the first term. The quantityP(B(t) ≥ a|Ta ≤ t) = 1∕2, sinceTa ≤ t means thattheprocesshit aatsometime spriorto tandbythesymmetryprincipleitisjustaslikely to be below as above. Formally, we are applying (18.11) withh= 0,k=∞ . This enables us to state the desired relationship P(Ta ≤ t) = 2P(B(t) > a). 298 AN INTRODUCTION TO STOCHASTIC PROCESSES SinceB(t)isnormalwithmean0andvariancet,weknowthat P(B(t) > a) = 1−Φ(a∕ √ t) where Φ is the c.d.f of the standard normal. Moreover, fora< 0, the distribution ofTa is, by symmetry, the same as thatT−a since we start the process at 0. Our final conclusion is then P(Ta ≤ t) = 2[1−Φ(|a|∕ √ t)]. Thisenablesustofindthedistributionofanotherrandomvariableofinterest.Let M(t)be the maximum value thatBassumes in the interval [0,t]. This will be of course a nonnegative random variable, since we start at 0. Fora> 0, we use continuity again to see thatM(t) will belargerthanorequalto aifandonlyiftheprocessreached asometimesintheinterval[0, t] which means thatTa ≤ t.S o P(M(t) ≥ a) = 2[1−Φ(|a|∕ √ t)]. This is an intuitively appealing result, since it says that for anya> 0, the probability that a standard Brownian motion takes a value greater thanaat any time before timetis exactly twice the probability that it takes a value greater thanaat timet. ∗18.7.4 Conditional distributions Given two timessandt, what is the conditional distribution ofBt given thatXs took a certain valueb? We consider the two cases. Ifs≤ t,thisisrelativelystraightforward.Wecanimaginetheprocessjuststartingallover again at time s except with a starting value ofbrather than 0. It is then easy to deduce that Ift≥ s,( B(t)|B(s) = b) is normal with meanband variance (t−s). (18.12) What ifs> t? Now we are asking for the distribution of a random variable conditional on future values and the procedure is more complex. Could we perhaps get the same answer except with variance= (s−t)? Obviously, not since this would be false fort= 0w h e r ew e knowbydefinitionthat B t takesthevalue0withcertainty.Itturnsoutthatwehavetomultiply these quantities byt∕s. Wewillworkwiththedensityfunctionsandlet fr standfor fB(r).Nowgiventhat B(s) = b, we will haveB(t) = a provided that the incrementB(s)−B(t) takes a value ofb−a.B y stationarity, this increment is distributed asBs−t. Then ft(a|Xs = b) = ft(a)fs−t(b−a) fs(b) , where we invoke the hypothesis of independent increments for the numerator. Letr= t∕s. Plugginginthevariousnormaldensitiesanddoingsomealgebra,thelefthandsidereducesto Ke x p (−(a−rb)2 r(s−t) ) for some constantKand we can then deduce that in place of (18.12) we have the following. Ift< s,( B(t)|B(s) = b) is normal with meanrband variancer(s−t), wherer= t∕s. NOTES AND REFERENCES 299 18.7.5 Brownian motion with drift ABrownianmotionprocessisamartingale.Wehavenotdefinedthispreciselyforacontinuous time process, but the idea is similar to that of the discrete time case. If a Brownian motion process takes the valuebat some point of time, then the expected value at any future point of time will beb, as the symmetry principle shows. One often wants to model situations where this is not the case and the quantity under study is growing in value. A simple model of this typeisaBrownianmotionwithadriftcoefficient 𝜇.Thisisaprocesswhichstillhasstationary and independent increments,X(t) is still normally distributed with variance𝜎 2t, but now, the mean ofX(t)i s𝜇trather than 0.In other words, on the average the quantity grows as a rate of 𝜇 per time period, so in any time period of lengthh, the quantity can be expected to increase by𝜇h. To state this precisely,X(t) is a Brownian motion with drift coefficient𝜇 and variance parameter 𝜎2 if and only if X(t) ∼ 𝜎B(t)+𝜇t. 18.7.6 Geometric Brownian motion Weareofteninterestedinquantitieswhichgrowataconstant relativeratherthan absoluterate ofgrowth.Anexampleisstockprices.Wemight,forexampleexpectastocktogrowinvalue atarateofsay5%peryear.Apossiblemodelinthiscaseis geometricBrownianmotion .This is a positive valued processX(t) such thatY(t), the logarithm ofX(t)i saB r o w n i a nm o t i o n with drift parameter𝜇 and variance parameter𝜎 2. That is, X(t) = eY(t) = e𝜇t+𝜎B(t). Note that eachX(t) will have a log-normal distribution. When𝜎 = 0,X(t)issimplytheamountofaquantitygrowingexponentiallyintimeatrate 𝜇.SoageometricBrownianmotioncanbeviewedasaprocessthathasanunderlyingpattern ofexponentialgrowth,witharateofgrowththatisnotconstantbutisitselfsubjecttorandom changes as given by a Brownian motion. This is a commonly used model for stock prices in modern finance theory and we will so apply it in Section 20.18. A quantity which is useful in many applications is the conditional expectation ofX(t), giving that the value ofX(s) = afor somes< t. To compute this, we first write X(t) = e Y(t) = eY(s)e(Y(t)−Y(s)) = X(s)e(Y(t)−Y(s)). We know thatY(t)−Y(s) is normal with mean (t−s)𝜇 and variance (t−s)𝜎2.S ou s i n gt h e expression given in Equation (A.58) E(X(t)|X(s) = a) = ae𝜇(t−s)+(t−s)𝜎2∕2. Notes and references Ross (2010) has a good basic introduction to stochastic processes, covering more material than we do here. The section on Poisson processes is based largely on Chapter 5 of that book.Hoel etal.(1972)giveasomewhatmoreadvancedtreatment.KemenyandSnell(1963) 300 AN INTRODUCTION TO STOCHASTIC PROCESSES provide extensive coverage of finite-state Markov chains. A proof for the sufficient condition givenfortheexistenceofalimitingdistributioncanbefoundthereinTheorem4.12.Mikosch (1998) has more material on Brownian motion, with applications to some of the material we discuss in Chapter 20. Exercises 18.1 Acertainstockhasapriceof100attime0.Atanytime k,let Mk denotetheaverage of the prices at times 0,1,…,k. The price at timek+1 will be eitherMk,Mk +20, or Mk −20, each with probability 1∕3. What is the distribution of the stock price at time 2? What is the expected value at time 2? Is this process a Markov process, martingale, submartingale, supermartingale? 18.2 Suppose that rblack balls andrwhite balls are distributed equally among two urns. At each trial, a ball is chosen randomly from each urn and put in the other urn. Let X n be the number of black balls in urn 1 at timen. Describe the transition function for this Markov chain. 18.3 Thepriceofacertainstockiseither9,10or11.Ifthepriceis9onanyday,thenext day’s price will either be 9 or 10 with equal probability. If the price is 11, the next day’s price will be either 11 or 10 with equal probability. If the price is 10, the next day’s price will be either 9, 10, or 11 with equal probability. (a) Model this as a Markov chain and write down the transition matrix. (b) If the price is 9 on Monday, what is the probability that it will be 11 on Friday? ∗(c) Findtheproportionofthetime,inthelongrun,thatthepricewillbeeachofthe three possible values. *18.4 Consider the Markov Chain on states 1 to 5 with transition matrix ⎛ ⎜ ⎜ ⎜ ⎜ ⎝ 2∕30 01 ∕30 01 ∕20 01 ∕2 1∕403 ∕40 0 1∕20 01 ∕20 03 ∕40 01 ∕4 ⎞ ⎟ ⎟ ⎟ ⎟ ⎠ . Decide whether each state is transient or recurrent. *18.5 Consider the Markov chain on states 1 to 4 with transition matrix ⎛ ⎜ ⎜ ⎜ ⎝ 1∕31 ∕31 ∕30 1∕31 ∕301 ∕3 1∕301 ∕31 ∕3 00 1 ∕21 ∕2 ⎞ ⎟ ⎟ ⎟ ⎠ . Decide whether each state is transient or recurrent. EXERCISES 301 *18.6 Consider the Markov chain on states 1 to 4 with transition matrix ⎛ ⎜ ⎜ ⎜ ⎝ 00 .80 .20 00 0 .30 .7 0.50 .50 0 0001 ⎞ ⎟ ⎟ ⎟ ⎠ . Decide whether each state is transient or recurrent. *18.7 For a two-state Markov Chain with transition matrix ( 1−pp q 1−q ) , find the limiting distribution in terms ofpandq *18.8 An indecisive diner enters a restaurant and is taken to a round table with five chairs. Unabletodecidewhichchairisbest,thedinerswitcheseveryminute,movingclock- wise with probabilityp and counterclockwise with probability 1−p. We can then consideraMarkovchain X n,thenumberofthechairoccupiedattime n,inminutes. Show that the condition ensuring a limiting distribution holds in this case. What is the limiting distribution? What can you say if there are four chairs rather than five? *18.9 A particle moves around a square with four vertices, numbered 0,1,2,3 going clockwise. It moves clockwise with probability 2∕3 and counterclockwise with probability 1∕3. Motion stops when the particle reaches vertex 3. Find the expected amountoftimethevertexwillspendatvertex2,startingfromeachofvertices,0,1,2. *18.10 ShowthatforeitheraperiodicorreducibleMarkovchain,givenanyinteger n,there exists some ordered pair (i,j) such that the entry in rowiand columnjofP n = 0. *18.11 Agolferhasaprobabilityof(1 −0.1n)ofmakingan n-footputt,where nislessthan 10. He adopts the following routine for practicing putts from three to seven feet. He begins with a three foot putt. Anytime he makes a putt, he tries one which is a foot longer and any time he misses, he goes back to a foot shorter putt. If he misses on the three-foot putt or makes the seven-foot putt, he repeats that distance. (a) Ifhecontinueshissessionforaverylongtime,whatisthemostlikelydistance that he will finish with? Estimate the probability of finishing at that distance. (b) Repeatpart(a)onlyassumingnowthatheispracticingputtsfromfourtoeight feet. 18.12 In a Poisson process, the probability that exactly one event will occur in any given hour is 3e −3. (a) What is the probability that exactly two events will occur in any 20-minute period? (b) Suppose you start observing the process at some point of time. What is the probability that it will be less than 10 minutes until an event occurs? 302 AN INTRODUCTION TO STOCHASTIC PROCESSES (c) Take a unit of time to be 1 hour. Starting at time 0, find the expectation and variance of the time of the fifth occurrence. 18.13 Suppose N(t) is a Poisson process such thatP[N(2) = 1] = 6e−6. (a) FindP[N(1) = 2]. (b) Identify the distributions (including parameters) ofWn, the associated waiting- time process, andTn, the time of thenth arrival. (c) FindP[(N(1) = 1 andN(3) = 3]. 18.14 Vehicles pass a certain marker on a highway in accordance with a Poisson process at the rate of 48 per hour; 25 % of the vehicles on the road are trucks and 75 % are cars. Suppose you observe this marker at a particular point of time. (a) Whatistheprobabilitythatexactlyfivevehicleswillpassinthefirst15minutes? (b) Giventhatexactlytwotruckspassedinthefirst5minutes,whatistheprobability that the fourth truck will pass in the first 10 minutes? (c) Given that exactly two trucks passed in the first 5 minutes, what is the expected value and the variance of the number of vehicles that passed in the first 10 minutes? 18.15 Thefollowinggivesinformationaboutthreecountingprocesses, N 1(t),N2(t),N3(t). Give reasons why each of these cannot be a Poisson process. (a) P[N 1(2)−N1(1) = 10] = 0.5,P[N1(3)−N1(2) = 10] = 0.4 (b) P[N2(1) = 1] = 0.5 P[N2(1) = 1 andN2(2) = 2] = 0.2 (c) P[N3(1) = 0] = 0.5, P[N3(2) = 1] = 0.3 18.16 Consider a Poisson process N(t) with rate 2 per time period. You are given that N(1.8) = 4. Using this information, answer each of the following. (a) What is the probability thatN(2.3) ≤ 6? (b) What is the probability that the fifth occurrence will be before time 2? (c) What is the expected time and the variance of the seventh occurrence? 18.17 At a subway station, eastbound trains and northbound trains arrive independently, both according to a Poisson process. On average, there is one eastbound train every 12 minutes and one northbound train every 8 minutes. Suppose you arrive at the subway station at a certain point of time and start observing trains. (a) What is the probability that exactly two eastbound trains will arrive in the first 24 minutes? (b) What is the probability that exactly two eastbound trains will arrive in the first 24minutesandexactlythreeeastboundtrainswillarriveinthefirst36minutes? EXERCISES 303 (c) Whatistheexpectedwaitingtime,inminutes,untilthefirsttrain(ofeithertype) arrives? (d) Whatistheprobabilitythatthefirsttraintoarriveiseastbound,andthenexttwo are northbound? (e) What is the probability that it will take at least 20 minutes for two northbound trains to arrive? 18.18 Events occur according to a Poisson process. Suppose that the expected number of occurrencesperhouris3,andlet Tn denotetimeinhoursofthe nthoccurrence.Find the expected value and variance ofT3 given that the first occurrence was at time 1 and the second occurrence was at time 2. 18.19 In a Poisson process, the probability is 0.60 that after an occurrence of the event, it willtakeatleast2monthsuntilthenextevent.Findtheprobabilitythatexactlyfour claims will occur within any 5-month period. 18.20 An insurer finds that out of a certain group of insured drivers, the accident rate over each 24-hour period rises from midnight to noon, and then declines until the following midnight. They decide that the number of accidents can be modelled by a nonhomogenous Poisson process where the intensity at timet is given by [1∕6−(12−t) 2∕1152], wheretis the number of hours since midnight. Find: (a) the expected number of daily accidents; (b) the probability that there will be exactly one accident between 6:00 a.m. and 6:00 p.m. 18.21 Suppose that X(t) is a Brownian motion with variance parameter 0.03. IfX(2) = 3, find the probability thatX(5) > 3.5. *18.22 Foracertainsecurity,each1unitinvestednowwillyieldtherandomamount e0.2B(t) at timetwhereB(t)isastandardBrowniannotion.Whatistheprobabilitythatsometime within the next five time periods, an initial investment will have doubled in value. *18.23 The price of a stock at timetis siven by S(t) = 15e𝜇t+0.3B(t), where B(t) is a standard Brownian motion. The quantity𝜇 is unknown. An investor buys the stock at time 3 and sells it for 20 a share at time 5. What is the probability that the investor lost money on this transaction? 19 Multi-state models 19.1 Introduction Multi-statemodelsareanattempttolookatavarietyoflifeinsuranceandannuitycontractsin aunifiedmanner,bymakinguseofMarkovprocesses.Asmotivation,takeanindividualnow age xand consider a two-state Markov chain, where the person is in state 0 (alive) or state 1 (deceased)atanytime.Lifeinsurancecontractsprovidebenefitsupontransferfromstate0to state 1, while life annuity contracts provide benefits as long as the process remains in state 0. Moregenerally,consideramultiple-decrementmodelfor( x)with mcausesoffailure.We canconsiderachainwith m+1states.State0meansthat(x )hasnotsuccumbedtoanycause and is often referred to as theactive state. Statej refers to having succumbed first to cause j. The insurance benefits discussed in Chapter 11 can be viewed as payments upon transfer from state 0 to other states. For still another example, consider a joint-life contract issued to (x) and (y). We can now take a chain with four states as illustrated in Figure 19.1. The arrows indicate that there are possible transitions from state 0 into state 1 or state 2, occasioned by the death of (y)o r(x) respectively, and then further transitions from state 1 or state 2 into state 3 when the second death occurs. The dotted line, showing a transition directly from state 0 to state 3, would not bepresentinouroriginalmodel,butwouldbethereifwewantedtoconsideracommonshock modelasdiscussedinSection17.6.Ajointlifeinsurancecanbeconsideredastwocontracts, one paying benefits upon transfer from state 0 to state 1, and the other paying benefits upon transferfromstate0tostate2.Ageneraltwo-lifeannuitycanbeconsideredasthreeseparate contracts, where theith contract,i= 0,1,2, pays benefits provided the process is in statei. This can be generalized to contracts involvingnlives where we will have 2 n states. Wecanofcourseimaginemoregeneralpatternsoftransition.Wemaywishtoinvestigate a more enriched multiple-decrement model where individuals can transfer between states several times. A disabled person might recover and re-enter the main group of lives. In our originalmodelweignoredwhathappenedtoalifeonceitleftthegroupforanycause,butwe FundamentalsofActuarialMathematics , Third Edition. S. David Promislow. © 2015 John Wiley & Sons, Ltd. Published 2015 by John Wiley & Sons, Ltd. Companion website: http://www.wiley.com/go/actuarial THE DISCRETE-TIME MODEL 305 Both alive State 0 (x) only State 1 (y) only State 2 Neither State 3 Figure 19.1 A two life multi-state model maywishtomodelthefactthatsomeoneleavingforacauseotherthandeathwillsubsequently die. There are many examples of insurance and annuity benefits applicable to this general case. When disability is one of the decrements, we may have a contract that pays benefits when a person becomes disabled, and then further benefits when a disabled person dies, and possibly additional payments that continue during disability. Indeed, a common provision in many life insurance policies is adisability premium waiverclause, which mean the person does not have to pay premiums during the time they are disabled. We view this as an annuity providing payments during a state of disability, which cease when there is a transfer back to an active state. These are only some of the possibilities, and we invite the reader to think of additional applications. In this chapter, we will investigate the general multi-state model. We first discuss the discrete-time model, where transitions between states can occur only at integer times. Fol- lowing that we deal with the more complicated continuous-time model, where we allow for transitions at arbitrary times. 19.2 The discrete-time model 19.2.1 Non-stationary Markov Chains The transitions in multi-state models are normally age-dependent. To handle this, we need to extend the concepts and notation that we introduced in Section 18.4.1 so that they apply to the non-stationary case. Foranon-stationaryMarkovChain,inplaceofthesingletransitionmatrix P,weneedfor each nonnegative integern, a matrixP n whoseijth entry ispij(n,n+1) The main quantities of interest are the probabilitiespij(k,n) of being in statejat timen conditional upon being in stateiat timek. In deriving (18.5), we do not need the fact that the matrices are the same, and the argument given is easily adapted to show that pij(k,k+n)i st h eijth entry ofPkPk+1 …Pk+n−1, (19.1) which reduces to (18.5) in the stationary case. Equation (18.6) now takes the form 𝜋k+n = 𝜋k PkPk+1 …Pk+n−1. 306 MULTI-STATE MODELS Unhealthy State 1 Deceased State 2 Healthy State 0 Figure 19.2 The healthy-unhealthy-deceased model For example, in our simple life-death chain the transition matrices are given by Pn = ( px+n qx+n 01 ) . In the multiple decrement example, the matrixPn will be similar to that above, except the first row will be (p(𝜏) x+n,q(1) x+n ,q(2) x+n ,…,q(m) x+n), and each other row will have 1 on the main diagonal and 0s elsewhere. We leave is as an exercise for the reader to write down the matrixPn in the joint two life case. We now introduce a multi-state model which is not covered by our previous cases of multiple lives or multiple-decrement theory, and to which we will refer to frequently in the rest of the chapter. It allows for atwo-way transitionbetween certain states, which is really wherethegeneralityofthemulti-statemodelisthemosthelpful.Thisisthechainasdepicted in Figure 19.2 where a life can be healthy or unhealthy, but we now allow for recovery from the unhealthy state. There are various applications, The unhealthy state could refer to being disabled, or sick, or a number of other possibilities for a state which we wish to distinguish somehow from the main group of lives, but which can transfer back to the main group. Example 19.1 Consider the model of Figure 19.2 and suppose that transition matrices for times 0, 1, 2 are given as follows; P 0 = ⎛ ⎜ ⎜ ⎝ 0.70 .20 .1 0.20 .60 .2 001 ⎞ ⎟ ⎟ ⎠ P 1 = ⎛ ⎜ ⎜ ⎝ 0.50 .30 .2 0.10 .60 .3 001 ⎞ ⎟ ⎟ ⎠ P 2 = ⎛ ⎜ ⎜ ⎝ 0.40 .30 .3 0.10 .50 .4 001 ⎞ ⎟ ⎟ ⎠ . What is the probability that a person who is healthy at time 0 will be unhealthy at time 2 and deceased at time 3? Solution. P 0P1 = ⎛ ⎜ ⎜ ⎝ 0.37 0.33 0.30 0.16 0.42 0.42 001 ⎞ ⎟ ⎟ ⎠ , THE DISCRETE-TIME MODEL 307 Using formula (19.1), the required probability equals p01(0,2)p12(2,3) = 0.33×0.4 = 0.132. 19.2.2 Discrete-time multi-state insurances Tomodelthegeneralcontractofthistype,wewilltakeaMarkovchainwithstatesnumbered from 0 toN. Suppose the process begins in stateaat time 0. Fix any two statesiand j, and consideracontractwhichpaysabenefitof b k attime k+1,providedatransferfromstate ito state joccurs between timekand timek+1. That is, the process was in stateiat timekand statejat timek+1. Notation If we have several different transfer benefits we will distinguish by writingbk as b(ij) k . In general, for quantities like benefits, premiums, reserves and expenses, we will let superscripts refer to states, and maintain our previous usage of subscripts to refer to time. This differs from our convention for probabilities where we use subscripts to refer to states, and write times in brackets. A convenient method of discussing multi-state benefits in the context of the stochastic model is to make use of indicator random variables. For each nonnegative integerk,l e tI k be therandomvariablethattakesthevaluesof0or1respectively,accordinglyasatransferfrom ito jhas not or has occurred between timekand k+1. The present value of the benefits on the contract is given by Z = ∞∑ k=1 bkv(k+1)Ik. (19.2) Since E(Ik) = pai(0,k)pij(k,k+1), (19.3) the actuarial present value is ∞∑ k=0 bkv(k+1)pai(0,k)pij(k,k+1). (19.4) Weagainhaveourfamiliarpatternwithasumofthree-termedfactors.Infact,wecanconsider each summand as a four-termed factor where the probability of receiving a payment is itself theproduct oftwofactors,theprobabilityoftransitiontostate ibytime kand theprobability of transition from stateito statejin the next period. Example 19.2 Consider the chain given in Example 19.1. A 3-year contract written on a healthy life provides for benefits at the end of the year of death. The death benefit is 3 if the person dies while healthy and 2 if the person dies while unhealthy. If interest is a constant 5%, find the actuarial present value. Solution. We can view this as two separate contracts, one paying 3 for transfer from state 0 to state 2, and the second paying 2 for transfer from state 1 to state 2. We use the matrices in 308 MULTI-STATE MODELS Example 19.1 and formula (19.1). For the first contract, p02(0,1) = 0.1, p00(0,1)p02(1,2) = 0.14, p00(0,2)p02(2,3) = 0.37×0.3 = 0.111, so that APV = 3[(0.1)(1.05)−1 +0.14(1.05)−2 +0.111(1.05)−3] = 0.9543. For the second contract, p01(0,1)p12(1,2) = 0.06, p01(0,2)p12(2,3) = 0.33×0.4 = 0.132 so that APV = 2[0.06)(1.05)−2 +0.132(1.05)−3] = 0.3369. Therefore, the total APV is 1.2912. Example 19.3 Take the same chain as in the previous example. Consider a contract that pays 1 at the end of any year of transfer from being healthy to being unhealthy, provided this occurs within the next 3 years. Find the expected value and variance of the benefits. Solution. We now have p 01(0,1) = 0.2, p00(0,1)p01(1,2) = 0.21 p00(0,2)p01(2,3) = 0.37×0.3 = 0.111, so that APV = 0.2(1.05)−1 +0.21(1.05)−2 +0.111(1.05)−3 = 0.4768. The variance calculation can not be done conveniently by adapting a version of formula (15.4). The situation here is different from the multiple decrement model where benefits are paidupontransitiontoanabsorbingstate.Inthiscase,therecanbebenefitspaidatmorethan onetime.Wemustinsteadusetheapproachtakeninthecurrentpaymentformulaforannuity variances. Note however that the covariance calculations will be more complicated than that givenby(15.11).Letting I k takethevalue1iftransferfromthehealthystatetotheunhealthy state occurs between timekandk+1, we have E(I0) = 0.2, E(I1) = 0.21, E(I2) = 0.111, E(I0I1) = E(I1I2) = 0, while E(I0I2) = p01(0,1)p10(1,2)p01(2,3) = 0.006, which is the probability of becoming unhealthy, recovering, and then becoming unhealthy again, thereby being paid at both time 1 and time 3. We can then calculate the covariances as Cov(I 0I1) =− 0.042, Cov(I 0I2) =− 0.0162 Cov(I 1I2) =− 0.02331, THE DISCRETE-TIME MODEL 309 leading to the calculation of the variance as (0.2)(0.8)(1.05)−2 +(0.21)(0.79)(1.05)−4 +(0.111)(0.889)(1.05)−6 −2[0.042(1.05)−3 +0.0162(1.05)−4 +0.02331(1.05)−5 = 0.2195. Suppose we wished to modify the contract of the previous example so that it paid off onlyonthe firstoccurrenceofbecomingunhealthy.Inoursimpleexample,thiscouldbedone readilyenoughbychangingtheprobabilityofabenefitpaymentattime3to0.105,subtracting 0.006,whichwehavecalculatedaboveastheprobabilityofthesecondoccurrence.Tohandle the problem in general, we need a more systematic approach. Consider a contract paying benefits upon the first transition from a statei 0 to a statej0. A method that works here ( as well for many other problems ) is to add additional states. The easiest way is to add a single new absorbing statez, and all transfers from statei0 to statej0 are redirected toz, and the contract is modified to pay upon a transfer fromi0 to z. To state this precisely, letp∗ denote probabilities in the augmented chain. To simplify the notation, fix a timenand writepij and p∗ ij for pij(n,n+1) andp∗ ij (n,n+1) respectively. Then p∗ is the same aspexcept that p∗ i 0j0 = 0, p∗ i 0z = pi0j0, p∗ zz = 1, p∗ zi = 0, i≠ z, p∗ iz = 0, i≠ z,i0 There is a more involved method which has the advantage that it preserves relationships betweenotherstatesthatmightbeneededinmorecomplexproblems.Thisistoadd N+1new statesi∗,i= 0,…,N,wherestate i∗ canbeconsideredasasortof‘clone’ofstate i.Transitions betweenun-starredstatesareasbeforeexceptthattransitionsfrom i0 toj0 aredirectedto j∗ 0 in placeof j0.Exceptforthat,therearenotransitionsbetweenthetwotypesofstates,starredand un-starred. Transitions between starred states are as given by the corresponding transitions betweentheirclones.Inthiscase, p ∗ i 0j0 = 0andforallorderedpairs(i, j)notequalto( i0,j0), p∗ ij = pij, p∗ i 0j∗ 0 = pi0j0, p∗ i ∗j∗ = pij, p∗ i ∗j = 0, p∗ ij ∗ = 0. (19.5) For example, if our original matrix for a two state model is given by P = ( a 1−a b 1−b ) , and we takei0 = 0,j0 = 1, the the matrix of the augmented chain, using the order 0,1,0∗,1∗ is P∗ = ⎛ ⎜ ⎜ ⎜ ⎝ a 00 1 −a b 1−b 00 00 a 1−a 00 b 1−b ⎞ ⎟ ⎟ ⎟ ⎠ . In addition to this, the provisions of the insurance are modified to pay upon a transfer from state i 0 to statej∗ 0 . After the first occurrence of such an event, there can never be a return to statei0, so subsequent occurrences are ruled out. 310 MULTI-STATE MODELS Remark Solving multi-state problems with large matrices can involve a great deal of cal- culation.Thereareanumberofcomputingpackagesthatcanassistwiththis.Oneconvenient example is the MMULT function of Excel ®. 19.2.3 Multi-state annuities We now look at annuity contracts associated with a chain. Suppose again that the individual begins in stateaat time 0. For a fixed statei, we can consider a contract that paysc k at time k provided the person is in statei at that time. LetIk now be the indicator random variable that takes the value of 1 or 0 according as the process is or is not in stateiat timek. Then the present value of the benefits is the random variable Y = ∞∑ k=0 ckv(k)Ik, from which we calculate immediately APV = ∞∑ k=0 ckv(k)pai(0,k). (19.6) To derive variances, we calculate for allm< n E(ImIn) = pai(0,m)pii(m,n). From this, we calculate the covariances and then proceed as in Example 19.3. Example 19.4 Consider again the chain given in Example 19.1. A contract on a healthy life provides for four yearly payments beginning at time 0. The amount of the payment is 1 if the person is healthy, or 2 if the person is unhealthy. If interest is a constant 5%, find the actuarial present value. Solution. We view this as two separate annuities. Contract 1 is for state 1 and contract 2 is for state 2. In addition to the matrixP 0P1 calculated in Example 19.1, we need P0P1P2 = ⎛ ⎜ ⎜ ⎝ 0.181 0.276 0.543 0.106 0.258 0.636 001 ⎞ ⎟ ⎟ ⎠ . The APV of contract 1 is 1+ 0.7 1.05 + 0.37 1.052 + 0.181 1.053 = 2.1586, and that of contract 2 is 2 ( 0.2 1.05 + 0.33 1.052 + 0.276 1.053 ) = 1.4564, giving a total APV of 3.6150. THE CONTINUOUS-TIME MODEL 311 Example 19.5 Suppose the contract in Example 19.2 is to be purchased by three level annual premiums of𝜋, which are payable only if the person is healthy. Calculate𝜋. Solution. Equating values of premiums and benefits, 𝜋(1+(1.05)−10.7+(1.05)−20.37) = 1.2912, which gives𝜋 = 0.6449. Thereareothertypesofannuitycontractsthatcanbehandledbythetrickofaddingstates. Suppose for example, a contract provides periodic annuity benefits starting when the process entersstate i 0 (orstartingattime0iftheprocessbeginsinstate i0),butwhichstopcompletely when the process leaves statei0, even if there is a subsequent return. The easiest approach here is to add an absorbing statezand all transitions from statei0 to other states are directed to statez, ensuring the process never returns toi0. If we want to preserve information about other transitions, the method is to add a clone for each state as we did in the insurance case. Transitions from statei 0 to any statej≠ i0 are redirected to the clonej∗, also ensuring that the process will never return to statei0. 19.3 The continuous-time model In a realistic situation, transitions between states need not occur at discrete intervals, but can happen at any point of time, and the benefits payable upon transition from one state to the another will be paid at the moment of transition. To model this situation, we need to consider continuous-time Markov processes. We still keep a finite number of states, but let time vary continuously. The definition parallels that which we make in Section 18.2, only we must consider all possible points of time. (We will follow the notational usage introduced in Section 18.5 of writing the time variable in brackets.) Definition19.1 Acontinuous-timeMarkovprocessisastochasticprocess X(t):0≤ t< ∞, whereeach X(t)isdiscrete,withthepropertythatgiven anysequence oftimes,0 ≤ t 0 < t1 < ⋯ < tn < tn+1, and any sequence (x0,x1,x2 …xn,xn+1) wherexi is a value ofX(ti)w eh a v e P(X(tn+1)=xn+1|X(tn)=xn,X(tn−1)=xn−1,…,X(t0)=x0)=P(X(tn+1)=xn+1|X(tn)=xn ) As before, for timess≤ tand any two statesi,jwe let pij(s,t) = P(X(t) = j|X(s) = i), which is the probability of reaching statejat timetwhen starting in stateiat times. 19.3.1 Forces of transition Withthecontinuoustimeframework,weareabletodefineanaloguesoftheforceofmortality inouroriginallife-deathmodel,andtheforcesofdecrementinthemultiple-decrementmodel. Aswehaveseeninearlierexamples,theprocessisoftenspecifiedbygivingtheseforces,and then the central problem is to use these to calculate the desired probabilities. 312 MULTI-STATE MODELS Definition 19.2 We define theforceoftransitionfromstateitostatejattimet as follows. Ifi≠ j 𝜇ij(t) = lim h→0+ pij(t,t+h) h , while 𝜇ii(t) = lim h→0+ pii(t,t+h))−1 h . Note that𝜇ii is always nonpostive, which may seem a bit strange at first, but it is easily explained if we consider, for example, the multiple-decrement case where state 0 refers to an active life. The negativity of𝜇 00 reflects the action of the other forces of decrement, which cause transferoutof state 0. An important fact is the following. Theorem 19.1 N∑ j=0 𝜇ij(t) = 0, foralliandt . Proof. Fix anyi and t. Refer to the littleo notation introduced in Section 18.3. From the definitions above, for all statesjand h> 0, pij(t,t+h) = h𝜇ij(t)+o(h),i≠ j, (19.7) and pii(t,t+h) = h𝜇ii(t)+o(h)+1. (19.8) Summing over alljgives 1 = h N∑ j=0 𝜇ij(t) = o(h)+1. Subtract 1 from each side, divide byh and take a limit ash approaches 0 to establish the conclusion. We now want to relate the forces of transition to the transition probabilities. This is much more complicated in our general setting than in the simple cases we looked at previously. Theorem 19.2 (Kolmogorov forward equations)Foranystatesi ,jandtimess < t 𝜕 𝜕tpij(s,t) = N∑ k=0 pik(s,t)𝜇kj(t) (19.9) Proof. Westartbyderivingthediscreteversionoftheaboveformula.Followingthereasoning used in Equation (18.4), we can deduce that for statesi,jtimes s< tand h> 0 pij(s,t+h) = N∑ k=0 pik(s,t)pkj(t,t+h), (19.10) THE CONTINUOUS-TIME MODEL 313 reflectingthefactthatinordertoreachstate jfromstate iinthetimeinterval s,t+hwemust first reach some statekby timetand then go from statekto statejin the nexthtime units. The system of equations given by (19.10) is known as theChapman-Kolmogorovequations. Substituting from (19.7) and (19.8) into the second factor of the summand, pij(s,t+h) = h N∑ k=0 pik(s,t)𝜇kj(t)+pij(s,t)+o(h). Subtractingpij(s,t) from both sides, dividing byhand taking a limit gives (19.9). We can give an intuitive explanation of (19.9) paralleling that given in the discrete case. View each side as a type of density function for reaching statejat timetwhen starting from state iat times. The right side reflects the fact that in order to accomplish this, we must first reach some statekat timetand then at that instant of time, transfer from statekto statej(or in the case thatk= j, not transfer back into some other state). Weviewthestatementofthetheoremasasystemofdifferentialequationsinthevariable t forafixedvalueof s.Theword‘forward’arisessinceourtransitionismovingforwardintime. There is a related system, the correspondingbackwardsequations, which will be developed in Exercise 19.6. There will normally be many solutions to a system of differential equations of this type. There will however often be a unique solution if we specify appropriate initial conditions. In our context, these conditions are the obvious requirements that for allt, p ij(t,t) = { 0, i≠ j 1, i= j. WehaveofcourseencounteredversionsoftheKolmogorovequationsbefore.Take s= 0. The simplest case withN = 1 gives Equation (8.15). In the multiple decrement model, the Kolmogorov equations produce Equation (11.10). These statements may not be completely obvious since the notation is somewhat different. It will be instructive for the reader to verify these claims. There is a nice matrix formulation of (19.9). Fixings,t we letP denote the transition matrixforthetimeinterval(s, t).Thisisthematrixwithanentryinthe ithrowand jthcolumn of p ij(s,t), as we had in the discrete case. We also have a corresponding matrixM in which the entry in theith row andjth column is𝜇ij(t). Mis often referred to as theintensitymatrix. We letP′ denote the matrix in which each entry is the partial derivative with respect tot of the corresponding entry inP. The Kolmogorov forward equations together with the initial conditions can then be expressed as P′ = PM, (19.11) and P= I fors= t. (19.12) Thefollowingisasimplebut usefulconsequence oftheforwardequations. Itgeneralizes a result from our life-death model, when we have a state, such as that of being alive, which cannot be entered from any other state. That is, for allj≠ iand s< t,w eh a v ep ji(s,t) = 0. We will refer to such a state as beinganti-absorbing. 314 MULTI-STATE MODELS In the following theorem, we adopt a formally weaker hypothesis involving forces rather than probabilities, which by the second statement turns out to be equivalent. Theorem 19.3 Letibeastatesuchthat 𝜇 ji(t) = 0forallj ≠ i,andallt.Then,fors < t, pii(s,t) = e∫ t s 𝜇ii(r)dr, and pji(s,t) = 0, forj ≠ i. Proof. From Theorem 19.2 we get 𝜕 𝜕tpji(s,t) = pji(s,t)𝜇ii(t), (19.13) afamiliardifferentialequationwhichwefirstencounteredin(8.15)andwhichiseasilysolved to give p ji(s,t) = K(s)e∫ t s 𝜇iird r forsomefunction K(s)independentof t,Since pii(s,s) = 1,and pji(s,s) = 0for j≠ i,wemust haveK(s) = 1i fj= ior 0 ifj≠ iand the conclusion follows. Another quantity of interest is the so-calledsojournprobability for a state. We define pii(s,t) = P[Xr = i,s≤ r≤ t|Xs = i]. Note that the sojourn probability will in general differ frompii(s,t), as the corresponding event requires the process to remain in stateifor an entire interval of time, rather than just at the endpoints. The two probabilities will however be the same for an anti-absorbing state. Since sojourn probabilities are featured prominently in the next section, we will attempt to find a way to evaluate them. Let qi(s,t) = pii(s,t)−pii(s,t) This is the probability that starting in stateiat times, the process is back in stateiat timet, having left stateisometime during the interval. Let vi(t) = lim h→0+ qi(t,t+h) h . Now we employ once again the technique of adding states. Given a process and a statei, we add a single new statei∗ which is a clone of statei. Transitions out of statei∗ follow the samepatternastransitionsoutofstate i,whiletransitionsthatoriginallycameintostate iwill be diverted to the cloned statei∗. Now to say that the new process is in stateiat a timesand also at a later timetmeans that it must have stayed there throughout as it cannot re-enter. In THE CONTINUOUS-TIME MODEL 315 addition, the forces of transition out of stateiremain the same in both processes. So we have that p∗ ii (s,t) = pii(s,t), 𝜇∗ ij = 𝜇ij, j≠ i,i∗ We do have to consider the quantity𝜇∗ ii∗.N o w p∗ ii ∗(s,t) = qi(s,t), since the new process will move from stateito statei∗ precisely when the original process moves from stateiback to stateihaving left stateiat some point. From Definition 19.2, 𝜇∗ ii∗(t) = 𝜈i(t). Applying Theorem 19.3 to stateiin the new process then leads to the formula pii(s,t) = e∫ t s (𝜇ii(r)−𝜈i(r))dr. This formula does not however allow one to calculate the term on the left exactly since𝜈i also involves sojourn probabilities. All we can say without further assumptions is that pii(s,t) ≤ e∫ t s 𝜇ii(r)dr. Suppose, however, that we have a process such that, for all statesiandt,h> 0 qi(t,t+h) = o(h). (19.14) Thisassumptionsaysthatitishighlyunlikelythatinasmalltimeintervalthereisamoveout of a state and back into it. It is the same idea that underlies a Poisson process as discussed in Section 18.3, where it was highly unlikely to have two occurrences of an event in a small time interval. When this assumption holds, we have thatv i(t) = 0 for allt and we can now state the following theorem. Theorem 19.4 Foraprocesssatisfying(19.14), p ii(s,t) = e∫ t s 𝜇ii(r)dr. Remark We can view the first statement in Theorem 19.3 as the special case of Theo- rem 19.4, whenqi is actually equal to 0. An alternative direct proof of Theorem 19.4 is outlined in Exercise 19.18. To summarize our conclusions in this section we have shown that given the forces of transition, we can in theory, solve a system of differential equations to arrive at the required probabilities that we are interested in for our applications. However, in practice, the solution canbedifficultorimpossibletoobtainexactly.Inthefollowingsubsectionswelookatvarious possibilities for obtaining or approximating the probabilities from the forces. 316 MULTI-STATE MODELS 19.3.2 Path-by-path analysis Assumethenthatwearegiven 𝜇ij(t)forall i,jandt,andwewanttodeterminetheprobability that starting in statei at any times, we will be in statej at timet. What we can do in a relatively straightforward manner is the following. Given any path going from stateito state j, we can derive a formula for the probability that starting in stateiat times, the process will be in statejat timet, having following the prescribed path exactly. Precisely, we are given a path 𝜋 which is a sequence of states,i0,i1,…,iM where i0 = iand iM = j. We then want the probability that the process will make a transition from stateito i1 followed by a transition from i1 to i2, followed by a transition fromi2 to i3 and so on, continuing in such a fashion, withoutvisitinganyotherstates,andfinallyreachingstate jatorbeforetime t,andremaining in statejuntil timet. Let us denote such a probability byp𝜋 i,j (s,t). These can be calculated by integration, using only the forces and sojourn probabilities. So assuming that (19.14) holds, we have reduced the problem to one of evaluating integrals. To illustrate, consider the simplest possible case of a path of length 1, that is𝜋 = (i,j). Then arguing much the same as we did in the single life and multiple decrement models we can write p 𝜋 i,j (s,t) = ∫ t s pii(s,r)𝜇i,j(r)pjj(r,t)dr. The above formula parallels formula (11.13) except that in that case all non-active states areabsorbing,soonecannotgetoutofthem,whichmeansthatthethirdfactorintheintegrand doesnotappearasitisequalto1.Torecap,theformulaaboveshowsthatinordertogofrom i tojalongtheone-steppath,threeeventshavetooccur.First,theprocessmustremaininstate iuntilsometime rbetweentime sandtime t.Thiseventhasprobability p ii(s,r).Itcannotfirst gotoanotherstateandreturn,sincethatwouldconstituteadifferentpathfromthegivenone. Secondly,theremustbeatransitiontostate jattime r.Wecanthinkoftheprobabilityofthis as being𝜇 i,j(r)dr. Finally, the process must remain in statejfrom timerto timet. It cannot leave statejand return as this would similarly constitute a different path. We get the required probability by integrating the product of these probabilities over all timesrbetweensandt. Asthepathlengthincreasestheformulagetsmorecomplicated.Consideratwo-steppath, 𝜋 = (i,k,j). Now we need a double integral. The formula is p𝜋 i,j (s,t) = ∫ t s ∫ t r1 pii(s,r1)𝜇ik(r1)pkk(r1,r2)𝜇kj(r2)pjj(r2,t)dr2 dr1. This looks extremely complex, but it simply records what are now five steps to meet the required conditions. The process must remain in statei until some timer1, then transfer to state k, then remain in statek until some timer2, then transfer to statej, and then finally remain in statejuntil timet. A similar calculation that arises for a given pathΠ, is to find the probability that starting in stateiat times, the process will transfer to statejbefore timethaving followed precisely the prescribed path. We get the same multiple integral as described above, except the final sojourn probability is omitted, since the process need only enter, but not necessarily remain in the final state on the path. Of course, when the final state is absorbing, the two problems are identical. THE CONTINUOUS-TIME MODEL 317 Example 19.6 Consider a heathy-unhealthy-deceased model as introduced in Figure 19.2. The forces of transition are constant with𝜇01 = 2, 𝜇02 = 1, 𝜇10 = 1, 𝜇12 = 3. Find the prob- ability that a healthy life will become unhealthy, and then die without recovering before time 1. Solution. We have𝜇 00 =− 3,𝜇11 =− 4.Thepathinquestionis 𝜋 = (0,1,2)andtherequired probability is p𝜋 02 (0,1) = ∫ 1 0 ∫ 1 r1 e−3r12e−4(r2−r1)3dr2dr1. The double integral evaluates to 6 4 [ 1−e−3 3 −e−4(e−1) ] = 0.4279. In general, a path of lengthn will require ann-dimensional integral with an integrand consisting of 2n+1 factors, so this can quickly become computationally infeasible. Aside fromthatthereisanotherdifficulty.Usually,whatwereallywantis pij(s,t).Insomecasesthis might be obtained by adding upp𝜋 i,j (s,t) for all possible paths𝜋 fromitoj. However, even in thesimplestofmodels,therecouldwellbeaninfinitenumberofsuchpathsandthisprocedure will not work. This will normally occur whenever there is a positive probability of two-way transitions, as in the healthy-unhealthy-deceased model where healthy people can recover. A person could go from a healthy state to the deceased state, by getting sick and dying, or getting sick, then recovering, then getting sick again and dying, or recovering twice before death, or any of a number of infinite possibilities. We need other approaches to calculating transition probabilities, which we discuss in the next two subsections. 19.3.3 Numerical approximation There are various ways to find a numerical approximation to the solution of the Kolmogorov equations. We will describe one approach that is fairly simple to implement. It is similar to Euler’s method, described in Section 8.9, for Thiele’s equation. For a fixed times, the goal is to approximate the probabilitiesp ij(s,t). Pick a small interval of timeh, and change our unit oftime,sothateachoriginaltimeunitisnow1 ∕hunits.(So,forexample,ifouroriginaltime unit was a year, andh= 1∕12, this changes the time unit to months.). We then essentially define a discrete time multi-state model, which will in generally be nonstationary. We will define one step transition probabilities for this model by simply looking at formulas (19.7) and(19.8),whichformedthebasisoftheKolmogorovequations,onlyomittingthe o(h)terms in each case. Since these terms becomes very small ash does, we can hope that for small enough h, we obtain a good approximation. We also switch the origin of time so that the original times is now time 0 and the original times+mh for an integerm is now timem. Wethentakeourapproximatingdiscretenon-stationaryMarkovchaintobethatinwhichthe transition probabilities, denoted bỹp,a r eg i v e nb y ̃p ij(m,m+1) = h𝜇ij(s+mh),i≠ j, ̃pii(m) = h𝜇ii(s+mh)+1. 318 MULTI-STATE MODELS We then obtain transition matrices (̃Pm) which have for an (i,j)th entry, the quantity ̃pij(m,m+1.) Anotherwayoflookingatthisisthatwesimplygetthematrix( ̃Pm)bytakingtheintensity matrixM(s+mh),multiplyingallnon-diagonalentriesby handadjustingthediagonalentries so that the rows add to 1. When t= s+mhfor some integermwe can then obtain approximations by pij(s,t) ≐ the (i,j)th entry of̃P0 ̃P1 ⋯ ̃Pm−1. Example 19.7 In a chain with three states, you are given that 𝜇01(t) = 𝜇10(t) = 0.09(1+t), 𝜇02(t) = 0.18(1+t), 𝜇12(t) = 0.09, 𝜇20(t) = 𝜇21(t) = 0. Find an approximation top01(1,2) using the method described above withh= 1∕3. Solution. We have M(1) = ⎛ ⎜ ⎜ ⎝ 0.46 0.18 0.36 0.18 073 0.09 001 ⎞ ⎟ ⎟ ⎠ . so that ̃P 0 = ⎛ ⎜ ⎜ ⎝ 0.82 0.06 0.12 0.06 0.91 0.03 001 ⎞ ⎟ ⎟ ⎠ and similarly ̃P 1 = ⎛ ⎜ ⎜ ⎝ 0.79 0.07 0.14 0.07 0.90 0.03 001 ⎞ ⎟ ⎟ ⎠ , ̃P 2 = ⎛ ⎜ ⎜ ⎝ 0.76 0.08 0.16 0.08 0.89 0.03 001 ⎞ ⎟ ⎟ ⎠ . Then the approximation top 01(1,2) is the entry in row 0 and column 1 of̃P0 ̃P1 ̃P2, which is 0.15131. 19.3.4 Stationary continuous time processes One case where we can make some progress in solving the Kolmogorov equations directly is thatofa stationaryprocessasdefinedinChapter18.Wethenhavethateach 𝜇 ij(t)isaconstant, denoted by𝜇ij. Moreover, the process will be determined by the quantitiespij(t) = pij(0,t) since the stationary condition implies thatpij(s,t) = pij(t−s). As an example, we can write down completely the solution for the general two-state stationary process. Suppose the intensity matrix is M = ( −𝜇𝜇 𝜈 −𝜈 ) . THE CONTINUOUS-TIME MODEL 319 Let𝜌(t) = e−t(𝜇+𝜈).Thenthetransitionmatrix P(t),whichhastheentryof pij(t)inthe ithrow andjth column, is given by P(t) = (𝜇 +𝜈)−1 ( 𝜇𝜌(t)+𝜈 −𝜇𝜌(t)+𝜇 −𝜈𝜌(t)+𝜈𝜈 𝜌 (t)+𝜇 ) (19.15) We can verify thatPsatisfies equations (19.11) and (19.12) by direct calculation, noting that d𝜌(t)∕dt) =− (𝜇 +𝜈)𝜌(t), and that𝜌(0) = 1. Forthegeneralstationaryprocess,solutionsto P′ = PMcanbegeneratedfrom eigenvec- torsof M. These are non-zero vectorsa = (a0,a1,…,aN)f o rw h i c h aM = 𝜆a for some constant𝜆 known as aneigenvalue of M. (The left hand side above is a matrix multiplicationinwhichweview aasa1 ×N+1matrix.Accordingtotheconventionwehave usedforthetransitionprobabilities,thevectorappearsontheleft.)Toobtaineigenvectors,we first find the eigenvalues as those constants for which the determinant of (𝜆I−M) = 0. This involvessolvingapolynomialequationin 𝜆.Wecanthensolvefortheresultingeigenvectors. Examples will follow. Note now that given any such eigenvectora and any fixedi, the matrixP that has all zero entries except for anith row of (a 0e𝜆t,a1e𝜆t…aNe𝜆t) is a solution toP′ = PM.T h i s follows since theijth entry ofPM will be𝜆aje𝜆t which is the derivative of the corresponding entry inP. Note next that given any finite number of solutions toP′ = PM, the linearity implies that any linear combination of these solutions will also be a solution. We therefore seek a linear combination that will satisfy the initial conditions. This can always be done in the case that we can findN linearly independent eigenvectors. We illustrate with a simple example. Example19.8 ForthedataasgiveninExample19.6,findtheprobabilitiesofthefollowing events. (a) A life now disabled will be active at time 0.5. (b) A life now active will be deceased at time 1. Solution. The intensity matrix is M = ⎛ ⎜ ⎜ ⎝ −32 1 1 −43 00 0 ⎞ ⎟ ⎟ ⎠ . We can see immediately that 0 is an eigenvalue with an eigenvector of (0,0,1). We can also note that since the rows ofP add up to 1, the rows ofM add to 0, and the last row ofP is (0,0,1), it is sufficient to solve the reduced systemP r′ = PrMr in which the superscript rindicates a matrix with the rowN and columnN removed. So we can consider eigenvectors of Mr = ( −32 1 −4 ) . 320 MULTI-STATE MODELS The determinant of𝜆I−Mr is 𝜆2 +7𝜆+10 giving eigenvalues of−5 and−2. Solving, we find respective eigenvectors of (1,−2) and (1,1). (Note that eigenvectors are not unique as they can be multiplied by any non-zero constant.). To satisfy the initial conditions, we want the first row ofPr to be the particular linear combination of the vectorsa = e−5t(1,−2) andb = e−2t(1,1) which gives the vector (1, 0) when t= 0. We solve this to get a first row vector of (1∕3)a+(2∕3)b. For the second row, we want a linear combination that gives the vector (0,1) whent= 0. We solve this to get a second row vector equal to (−1∕3)a+(1∕3)b. Transition probabilities are then given by p00(0,t) = 1 3e−5t + 2 3e−2t, p01(0,t) =− 2 3e−5t + 2 3e−2t, p10(0,t) =− 1 3e−5t + 1 3e−2t, p11(0,t) = 2 3e−5t + 1 3e−2t. The answers to the particular questions asked are (a) p10(0,0.5) = 0.0953. (b) 1−p00(0,1) −p01(0,1) = 0.8218. (The answer to part (b) is of course larger than that of Example 19.6 since here we allow for an arbitrary number of recoveries from being sick, as well as death from a healthy state. 19.3.5 Some methods for non-stationary processes The above procedure can also be used solve the problem of determining probabilities from the forces, when these are ‘piecewise constant’. We can find transition matrices for each time interval on which the forces are constant, and then use the discrete time technique. The followingsimpleexampleillustratestheprocedure.Itinvolvesonlytwotimeintervalsbutthe method can easily be extended. Example 19.9 For a two-state model, we have forces of transition given by 𝜇 01(0,t) = { 1, if 0 ≤ t< 2, 2i f 2 ≤ t< 3. 𝜇10(0,t) = { 2, if 0 ≤ t< 2, 3i f 2 ≤ t< 3. Find the probability that the process will be in state 1 at time 2.5 given that it is is state 0 at time 0. Solution. p 01(0,2.5) = p00(0,2)p01(2,2.5) +p01(0,2)p11(2,2.5). THE CONTINUOUS-TIME MODEL 321 From the matrix (19.15), we have p00(0,2) = 2+e−6 3 , p01(0,2) = 1−e−6 3 p01(2,2.5) = 2−2e−2.5 5 , p11(2,2.5) = 2+3e−2.5 5 Substituting, the required probability is 0.39446. One method for handling a perfectly general chain is to approximate it by a piecewise continuousone,andthenusethemethodoutlinedabove.Todoso,onecanchooseasmalltime unit, and approximate the forces of transition by replacing them with those that are constant on each time interval, possibly using the midpoint value. 19.3.6 Extension of the common shock model Asmentionedintheintroduction,multiplelifetheory,multipledecrementtheory,orthemore general model discussed in Chapter 17 can all be looked at in a multi-state framework. In the standard cases where failure times are independent, this does not normally provide any advantageovertheconventionaltreatmentsthatwehavedescribedpreviously.Themulti-state approach,however,canbeusefulinmodellingcertaintypesofdependence.Asanillustration, we will revisit the common shock model of Section 17.5 and discuss some possible refine- ments.Weconfinetheanalysistothecaseoftwotypesoffailure,thefirstwithfailuretimeequal tomin(T ∗ 1,Z)andthesecondwithfailuretimeequalto min(T∗ 2,Z).Failureforthetwotypesis dependentduetothecommonshockbuttherewillbeadditionalsourcesofdependencewhen T ∗ 1 andT∗ 2 arethemselvesnotindependent.Anexampleofthisisthe‘broken-heart’syndrome mentioned in Section 10.2 where one type of failure can hasten failure of the other type. We consider a multi-state model with four states, as given in Figure 19.1, but adapted to cover general failure times. State 0 means that neither type of failure has occurred, state 1 meansthatonlythesecondcauseoffailurehasoccurred,state2meansthatonlythefirsttype offailurehasoccurred,andstate3meansthatbothtypesoffailurehaveoccurred.(Thestates 1 and 2 then are labelled by the number of the surviving type.) Let𝜇 i(t) denote the hazard function forT∗ i , and𝜌(t) denote the hazard function forZ. Suppose that forces of failure are of the form. 𝜇01(t) = 𝜇2(t), 𝜇02(t) = 𝜇1(t), 𝜇03(t) = 𝜌(t), 𝜇13(t) = 𝜇1(t)+𝜌(t)+𝜖1(t), 𝜇23(t) = 𝜇2(t)+𝜌(t)+𝜖2(t) for some functions𝜖1(t) and𝜖2(t). When these are 0, we have precisely the common shock model. In this more general setting, we use𝜖i(t) to build in some dependence betweenT∗ 1 andT∗ 2 by letting the distributions change after the first failure. The ‘broken-heart syndrome’ wouldcallforpositivevaluesof 𝜖i(t).Ontheotherhand,theremaybecaseswhereweassume that the prospects for failure typeiimproves after the first failure, and we would reflect this w i t han e g a t i v e𝜖i(t). Probabilities in this model can all be calculated by the procedure of Section 19.3.2 since thereareatmostthreepathsbetweenanytwostates.Moreover,sincenostatecanbere-entered once the process leaves it, we can use Theorem 19.4 for the sojourn probabilities. 322 MULTI-STATE MODELS For the following examples we assume that all forces areconstant. Example 19.10 Calculate the probability that at timen, the first type of failure will have occurred but not the second. Solution. From Theorem 19.1, 𝜇 00 =− (𝜇1 +𝜇2 +𝜌), 𝜇22 =− (𝜇2 +𝜌+𝜖2). We want p02(0,n) = ∫ n 0 p00(s)𝜇02(s)p22(s,n)ds= 𝜇1 ∫ n 0 e−(𝜇1+𝜇2+𝜌)se−(𝜇2+𝜌+𝜖2)(n−s) ds. The integral is easily evaluated to give p02(0,n) = 𝜇1 [ e−(𝜇2+𝜌+𝜖2)n −e−(𝜇1+𝜇2+𝜌)n 𝜇1 −𝜖2 ] . (19.16) Asacheck,supposethat 𝜖1 = 𝜖2 = 𝜌 = 0,sowesimplyhavetwoindependentexponential failure times. The answer reduces to e−n𝜇2(1−e−n𝜇1), which is just the probability that the first type of failure occurred before timenmultiplied by the probability that the second type did not occur before timen. Example 19.11 Find the probability that the second cause of failure occurs before timen and after the first cause of failure. Solution. IfΠis the path (0,2,3) we want, p Π 03 (0,n) = ∫ n 0 p02(0,t)(𝜇2 +𝜌+𝜖2)dt, (aspecialcaseofformula(19.17)inthenextsection.)Substitutingfrom(19.16)andintegrat- ing, the answer is 𝜇 1(𝜇2 +𝜌+𝜖2) 𝜇1 −𝜖2 [ 1−e−n(𝜇2+𝜌+𝜖2) 𝜇2 +𝜌+𝜖2 − 1−e−n(𝜇1+𝜇2+𝜌) 𝜇1 +𝜇2 +𝜌 ] When 𝜖2 = 0 we recover the answer of Example 17.12 (a). 19.3.7 Insurance and annuity applications in continuous time Consider an insurance contract that provides for two fixed statesj and k, payments at the moment of transfer whenever a transfer occurs from statejto statek. The amount paid for a THE CONTINUOUS-TIME MODEL 323 transferattime twillbe bt (denotedby b(jk) t ifwewishtodistinguishbetweenseveraltransfer benefits).Supposetheprocessisinstate iattime0.Thentheactuarialpresentvalueparallels the result we saw in the multiple decrement case and is given by ∫ ∞ 0 btv(t)pij(0,t)𝜇jk(t)dt. If benefits are to be paid on only the first transfer from statejto statek, then this can be handled by adding new states, exactly as we did in the discrete case. Similarly, for a contract that provides payments made continuously at the periodic rate of ct (writtenas c(j) t ifneededtodistinguishstates)attime t,providedtheprocessisinstate jthe present value is given ∫ ∞ 0 ctv(t)pij(0,t)dt. Some annuities may make use of the sojourn probabilities. Suppose, for example, that at time 0, the process is in statei, and payments at the periodic rate ofct are paid as long as the processremainsinstate i.Allpaymentsstopuponthefirstexitfromstate i.Thepresentvalue is given by ∫ ∞ 0 ctv(t)pii(0,t)dt. It is possible to fit the continuous time insurances and annuities into a stochastic model, and calculate variances aswell as expected values, aswe didinthediscretecase. Wewillnot pursuethishere.Ingeneral,thiswillrequireintegrationoffunctionswhosevaluesarerandom variables rather than definite numbers. This presents both theoretical and computational difficulties. More general probabilities can be deduced from the insurance formulas by the standard method we described in Section 10.8.4 of taking zero interest and benefit functions that take the value 1 or 0. For example suppose the process is in statei. The probability that withinn years it will at some point make a transfer from statejto statekis given by ∫ n 0 pij(0,t)𝜇jk(t)dt. (19.17) Example 19.12 An insurance contract based on the model in Example 19.8 provides for a benefit at timet of e.04t provided that a person now healthy dies while unhealthy. The force of interest is a constant 5%. Find the actuarial present value. Solution. This is given directly by ∫ ∞ 0 e0.04te−0.05tp01(0,t)𝜇12(t)dt= 3∫ ∞ 0 e−.01t ( 2e−2t −2e−5t 3 ) dt= 0.596. The following is an example which can be thought of as a generalization of contingent insurances. Suppose we have a path of statesΠ=i 0i1 …iM as described in Section 19.3.2. At time 0, the process is in statei0 and an insurance contract provides a payment of 1 at the momentthattheprocessentersstate iM,havingpreviouslyfollowedpreciselythepath Π.F or 324 MULTI-STATE MODELS example, consider the chain covering the two lives (x) and (y) as given in the introduction and the pathΠ= {0,1,3}. The contract based on this path is the second death-contingent insurance paying upon the death of (x) if this occurs after the death of (y). Example 19.13 Suppose that the force of interest and all forces of transition are constant. Find a formula for̄AΠ, the actuarial present value of the insurance based on the pathΠ. Solution. Tosimplifynotation,let vj = 𝜇ijij+1 and𝜇j =− 𝜇ijij Then,followingtheexplanation for the calculation of probabilities in section 19.3.2, we can set up the following multiple integral in whichr j denotes the time of transfer from stateij−1 to stateij. ̄AΠ = ∫ ∞ 0 ∫ ∞ r1 …∫ ∞ rn v0v1 …vM−1e𝜇0r1e𝜇1(r2−r1) ⋯e𝜇M(rN−rM−1)e−𝛿rMdrNdrM−1 ⋯dr1. The integration can be carried out in a straightforward manner to give ̄AΠ = v0v1 …vM (𝜇0 +𝛿)(𝜇1 +𝛿)⋯(𝜇M−1 +𝛿). Theformulaisquiteeasytorememberandtocomputefrom.Thenumeratoristheproduct of all the forces of transition along the path. The denominator is a product of factors, one for eachstateonthepath,exceptforthefinalone.Eachsuchfactoristhesumofalloutgoingforces fromthatstateplus 𝛿.Wecantake 𝛿 = 0togivetheprobabilitythatthepath Πwillbefollowed. NotethatFormula(8.28),Example10.5andFormula(10.25)withsubstitutionfromthese first two are special cases of the above. 19.4 Recursion and differential equations for multi-state reserves The basic idea of calculating reserves for multi-state contracts is the same as we have seen, exceptthatthereservemustbecalculatedforeachstate,(anideawehavealreadyencountered in Section 10.4.2). We let kV(j) denote the reserve at timek when in statej. Given that the process is in a certain state, the reserve at any time is as usual, the present value of future benefits less the present value of future premiums. Recursion and differential equation formulas can become more complicated. We start first by looking at the discrete time case, where we assume transfer benefits are payable at the end of the period of transfer. Example19.14 Considerthehealthy-unhealthy-deceasedmodel,andsupposethatwehave a stationary chain with transition matrix P = ⎛ ⎜ ⎜ ⎝ 0.60 .20 .2 0.20 .60 .2 001 ⎞ ⎟ ⎟ ⎠ . RECURSION AND DIFFERENTIAL EQUATIONS FOR MULTI-STATE RESERVES 325 An insurance contract provides for payments of 2 when a healthy life becomes disabled, 3 when a healthy life dies, and 4 when a disabled life dies. All benefits are paid at the end of the year of the change of state. This is paid for by a single premium. Assume the interest rate=0. Find the reserves at timek,f o rk> 0. Solution. Thiscanbedonebyrecursion,butwewillfirstdoitinamorecomplicatedway,so thattheeaseoftherecursionformulamethodcanbebetterappreciated.Wecouldconsiderthis as three separate contracts, but it is just as easy to do it all at once. In view of the stationarity, the reserve at timekfork> 0 will be independent of the time. It will however depend on the state at timek.L e ta= kV(0) and b= kV(1). In view of the single premium,awill just equal the present value of future benefits given the state is 0. Therefore, summing over all possible transfer timesj,w eh a v e a= 2 ∞∑ j=0 p00(0,j)p01 +3 ∞∑ j=0 p00(0,j)p02 +4 ∞∑ j=0 p01(0,j)p12. (19.18) Now, in this case, the matrixP is of a particular simple form. Namely P = ⎛ ⎜ ⎜ ⎜ ⎝ c+d 2 c−d 2 1−c c−d 2 c+d 2 1−c 00 1 ⎞ ⎟ ⎟ ⎟ ⎠ . It is not hard to see that P k = ⎛ ⎜ ⎜ ⎜ ⎜ ⎝ ck+dk 2 ck−dk 2 1−ck ck−dk 2 ck+dk 2 1−ck 001 ⎞ ⎟ ⎟ ⎟ ⎟ ⎠ , Inthepresentcase,wehave c= 0.8,d= 0.4.Substitutingin(19.18),weeasilysumtheinfinite geometric progressions to get a= 21 2 ( 1 0.2 + 1 0.6 ) 0.2+21 2 ( 1 0.2 + 1 0.6 ) 0.2+41 2 ( 1 0.2 − 1 0.6 ) 0.2 = 14 3 , We will leave it to the reader to verify similarly thatb= 13∕3. Wenowlookatgeneralrecursionformulas.Supposewehaveacontractprovidingforeach i,j,benefitsof b(ij) k attime k+1providingthereisatransferfromstate itostate jbetweentime kand timek+1. If we want to include expenses of payment, these can be incorporated into the transfer benefits. We can assume that for each statei,w eh a v eb(ii) k = 0, since payments for remaining in a state will be handled by annuity type benefits. We let𝜋(i) k denote thenet paymentcollectedattime kassumingtheprocessisinstate i.Thesewillbethepremiumless 326 MULTI-STATE MODELS any annuity payments paid out and less any expenses if applicable. Then if the system is in stateiat timek, k+1V(i) = ( kV(i) +𝜋(i) k ) − ∑ j≠i (pij(k,k+1) ( b(ij) k +k+1V(j) −k+1V(i) ) (19.19) Notethatthisissomewhatmorecomplicatedthanpreviousrecursionformulas,wherethe transfer was always to the deceased state, for which no reserves are required. In this more general case, upon transfer from stateito statej, we pay out the net amount at risk, and then also must set up the reserve required for statej, necessitating the extra term of k+1V(j) above. The following rearranged version of the recursion formula is instructive as well as being well suited to computation. For each statei, ( kV(i) +𝜋(i) k ) (1+ik) = ∑ j pij(k,k+1) ( b(ij) k +k+1V(j) ) . This simply says that the amount accumulated at the end of a period must provide for all the benefits paid out at that time, as well as all the reserves that are needed in the various states. Solving this general recursion to get reserves is not as straightforward as the cases we have previously looked at where reserves were all 0 except for one state. In general, we need to solve a system of linear equations rather than a single equation. Example 19.15 Solve the previous example by recursion. Solution. Substitute in (19.19) to get a= a−0.2(2+b−a)−0.2(3−a), b= b−0.2(a−b)−0.2(4−b), (19.20) which simplifies to 0.4a−.0.2b= 1, −0.2a+0.4b= 0.8, and this is solved to give a= 14∕3, b= 13∕3. Consideravariationontheabovewherealevelpremiumof 𝜋 ispaidaslongastheperson ishealthy.Wenowneeda 𝜋 ontherighthandsideofthefirstequationin(19.20)andsolving will lead to a= 14−10𝜋 3 b= 13−5𝜋 3 . Interestinglyenough,wecannowcalculatetheequivalenceprinciplepremium.Inviewof the stationarity, this must be the value of𝜋 which makesa= 0. This will be𝜋 = 7∕5 and the corresponding reserve for the unhealthy lives will be 2. The value of 7∕5 can also be verified PROFIT TESTING IN MULTI-STATE MODELS 327 directly by dividing the benefit present value of 14∕3 by the value of an annuity of 1 per year while the life is healthy. This is calculated as in Example 19.19 by summing the appropriate geometricprogressiontogive(1 ∕2)(1∕.2+1∕.6) = 10∕3.Thedisabledstatereserveof2can always be verified since 2=0.6(2) + 0.2(4), showing that 2 will provide for the same reserve for those who remain disabled and a death benefit of 4 for those who die. Inthecontinuouscase,wehaveThiele’sdifferentialequationsforthemulti-statereserves. Consider a contract providing benefits, including accompanying expense, ofb (ij) t at timet for transfer from statei to statej. Net payments are are collected continuously, where the periodic rate of payment at timetis 𝜋(i) t when the system is in statei. We then have a system of differential equations, one for each statei, as follows. d dttV(i) = 𝛿tt V(i) +𝜋(i) t − ∑ j≠i 𝜇ij(t) ( b(ij) t +tV(j) −tV(i) ) which we can also write as d dttV(i) = 𝛿tt V(i) +𝜋(i) t − ∑ j 𝜇ij(t) ( b(ij) t +tV(j) ) , (19.21) by invoking Theorem 19.1. 19.5 Profit testing in multi-state models WenowrevisitSection12.4.3andadaptittothemulti-statecase.Aswedidinthatsection,we will include notation for all the expenses, rather than incorporating them into other symbols. Suppose we have a general insurance contract based on a discrete time multi-state model providing for benefits ofb (ij) k at timek+1 for transfer from statei to statej between time k and k+1 and annuity benefits ofc(i) k paid at timek when in statei. We have a profit test basis at timekconsisting of premiums𝜋(i) k , percentage of premium expenses ofr(i) k , periodic expenses ofe(i) k , all paid at timek when in statei, and in addition a transition matrixPk, interest ratesik, and transfer expenses ofe(ij) k paid at timek+1 for a transfer from stateito state jbetween timekand timek+1. The profit testing procedure will conform precisely to the principles outlined in the single-life case. The profit for the period running from timek tok+1, for a policy in stateiat timekis given by Pr(i) k+1 = ( kV(i) +𝜋(i) k −c(i) k −e(i) k ) (1+ik)− ∑ j (pij(k,k+1) [ (b(ij) k +e(ij) k +k+1V(j) ] Note that this is just a straightforward generalization of the formula (12.4b) with the modification that we must provide reserves for the states transferred into, as noted in the Section 19.4. Example 19.16 An insurance policy issued to a healthy life provides for death benefits at the end of the year of death of 100000 if death occurs while healthy or 70000 if death occurs while unhealthy. In addition there are yearly payments of 30000 provided the insured 328 MULTI-STATE MODELS is unhealthy. You are given reserves as follows, with state 0 being healthy and state 1 being unhealthy. 5V(0) = 25000, 5V(1) = 75000, 6V(0) = 30000 6V(1) = 60000 The profit test basis has annual premium of 15,000 paid while healthy, an interest rate of 10%, periodic expenses paid each year of 200 if healthy or 400 if unhealthy, a transition matrix for the year running from time 5 to time 6 as the matrixP 0 of Example 19.1, and expenses for paying a death claim of 1% of the face amount. Find the profit Pr6 both when the policyholder is healthy at the end of 5 years and when the policy holder is unhealthy at the end of 5 years. Solution. Pr (0) 6 = (25000 +15000 −200)(1.1)−0.7(30000) −0.2(60000) −0.1(100000(1.01)=680. Pr(1) 6 = (75000 −30000 −400)(1.1)−0.2(30000) −0.6(60000) −0.2(70000(1.01) =−7080 19.6 Semi-Markov models In this section, we discuss briefly the problem that arises when the Markov condition does not hold. This is a frequent occurrence, since this will happen whenever the probabilities of movement from one state to another depend on the length of time elapsed since entering the current state. These are known assemi-Markov models. A typical example is found in the healthy-unhealthy-deceasedmodel.Thelikelihoodofeitherrecoveryordeathforaunhealthy person will certainly depend on how long they have been in the unhealthy state, a fact which was not considered in our previous models. We will not discuss these in detail. One method of approach is to approximate the semi- MarkovmodelbyaMarkovchain,bydividingstatesupintoseveralsub-states.Forexample, in place of a single statehfor being unhealthy we could have statesh 1,h2,…,hk where the only transitions with positive probability between this collection of states is fromhi to state hi+1. Therefore, a higher subscript is indicative of a longer period of being unhealthy. The probabilitiesoftransitiontobothrecoveryanddeathcouldthendifferbetweenthesub-states. Notes and references For a more detailed account of multi-state theory than we have presented here, see Norberg (2008). Transition probabilities, which involve four variables, can be written in a myriad of ways, and various choices of notation are found in different works. In much of the actuarial literature, the standard form of notation is maintained. The basic symbol used is tpij x to refer to the probability that a life now agexand in stateiwill be in statejat timet. In the notation ofthischapter,wewouldconsidertheage(x )fixedandwritetheaboveprobabilityas pij(0,t), following the typical usage found in much of the probability theory literature. The actuarial notation has the advantage of following classical actuarial usage for probabilities, but seems EXERCISES 329 restrictedtomodelswhichrefertoabodyoflives,andnotdirectlyapplicabletomoregeneral situations. See Jones (1997) for an interesting application of multi-state models. Exercises 19.1 RedoExample19.2,butassumenowthatinthefirstyear,theprobabilitythatahealthy person will become ill is 0.3 rather than 0.2, and the probability that they will remain healthy is 0.6 rather than 0.7. 19.2 Consider a Markov chain with transition matrices for the first three time periods given by P 0 = ⎛ ⎜ ⎜ ⎝ 1∕31 ∕31 ∕3 1∕31 ∕31 ∕3 1∕31 ∕31 ∕3 ⎞ ⎟ ⎟ ⎠ , P 1 = ⎛ ⎜ ⎜ ⎝ 1∕21 ∕20 01 ∕21 ∕2 1∕21 ∕20 ⎞ ⎟ ⎟ ⎠ , P 2 = ⎛ ⎜ ⎜ ⎝ 1∕21 ∕41 ∕4 1∕21 ∕41 ∕4 1∕21 ∕41 ∕4 ⎞ ⎟ ⎟ ⎠ . Interest rates are given byi 0 = 0.05, i1 = 0.06, i2 = 0.07. Find the APV of each of the following contracts. (a) Attime0,theprocessisinstate0.Acontractprovidesforpaymentsattheendof the period of a transfer from state 0 to state 1 if this occurs within three periods. The payment is 100 in the first year, 200 in the second year, and 300 in the third year. Note that more than one payment can be made. (b) At time 0 the process is in state 0. A contract provides for a payment of 1000 at timek,k= 0,1,2,3, provided the process is in state 0 at timek. 19.3 A transition matrix for a three-state homogeneous Markov chain is given by P = ⎛ ⎜ ⎜ ⎝ 0.40 .30 .3 0.70 .20 .1 0.20 .70 .1 ⎞ ⎟ ⎟ ⎠ Aprocessstartsinstate0.Annuitycontract1providesforperiodicpayments,provided the process is in state 1. Annuity contract 2 provides for periodic payments which starts at the end of the period in which process enters state 2, but stop completely upon exit from state 2 and do not begin again, even upon subsequent return. For each contract, find the probability that a payment is made at time 3. 19.4 A transition matrix for a four-state homogeneous Markov chain is given by P = ⎛ ⎜ ⎜ ⎜ ⎝ 0.10 .20 .30 .4 0.30 .10 .30 .3 0.50 .20 .10 .2 0.40 .30 .20 .1 ⎞ ⎟ ⎟ ⎟ ⎠ . 330 MULTI-STATE MODELS Attime0,theprocessisinstate0.Acontract provides forpayments attheendofthe year of thefirsttransition to state 3 from any other state, provide this occurs within 4 years. The amount of the payment is 100 if the transition is from state 0, 200 if the transition is from state 1, or 300 if the transition is from state 2. Level net annual premiums are paid beginning at time 0, and continuing until the time of transition to state 3. The interest rate is a constant 5%. (a) Find the annual premium. (b) Find the reserve at time 2, assuming the process is at that time in (i) state 0, (ii) state 1, (iii) state 2. 19.5 Verify that the Kolmogorov forward equations withs= 0 give Equation (8.15) in the case of a single life, or Equation (11.10) in the multiple decrement model. 19.6 Derive the Kolmogorov backwards equations. For timess≤ t, 𝜕 𝜕spi,j(s,t) =− N∑ k=1 𝜇i,k(s)pk,j(s,t). 19.7 Redo Examples 19.8 and 19.12 now given that𝜇01 = 1, 𝜇02 = 3, 𝜇10 = 2, 𝜇12 = 3. 19.8 For a two-state chain, derive the matrix (19.15) directly by using eigenvalues and eigenvectors. 19.9 In a two-state model, the forces of transition are given by𝜇01(0,t) = 1 for alltwhile 𝜇10(0,t) = { 1i f 0 ≤ t< 1, 3i f 1 ≤ t. Find the probability that the process will be in state 0 at time 2.4 given that it is in s t a t e1a tt i m e1 . 4 . 19.10 A stationary Markov chain has the following transition matrix. P = ⎛ ⎜ ⎜ ⎝ 0.60 .10 .3 0.20 .50 .3 0.10 .30 .6 ⎞ ⎟ ⎟ ⎠ . Interestisaconstant5%.FindtheAPVofthefollowingcontractsbasedonthischain, both issued when the process is in state 0. (a) An temporary annuity provides 100 per year, beginning when the process enters state 1 and stopping when the process leaves state 1. Payments do not resume upon re-entry to state 1. The last possible payment is at the end of 4 years. (b) A 7-year term insurance pays 100 at the end of the year of thefirsttransfer from state 0 to state 2, provided that that prior to this time there was a transfer from state 1 to state 0. If a transfer from state 0 to state 2 occurs before a transfer from state 1 to state 0, then nothing is paid. EXERCISES 331 19.11 Consider the chain of Figure 19.1. Suppose that (x) andyare two independent lives, where x is subject to a constant force of mortalitya and (y) is subject to a constant force of mortalityb. (a) Write down the reduced intensity matrixMr, (as defined in Example 19.8). (b) Writedownthereducedtransitionmatrix Pr(t)directlyfromChapter10formulas. (c) Find the eigenvalues and eigenvectors ofMr. (d) Verify that the rows ofPr(t) are linear combinations of vectors of the form e𝜆t(a1,a2,a3)where 𝜆isaneigenvalueof Mr and(a1,a2,a3)isthecorresponding eigenvector. 19.12 Consider the chain of Figure 19.1. We do not necessarily assume independence. You are given the constant intensities𝜇01 = b, 𝜇02 = a, 𝜇13 = c, 𝜇23 = d. (a) Given any positiveu, v, find the probability that (x) survives u years and (y) survivesvyears, in terms ofa,b,c,d. (b) Show thatT(x) andT(y) are independent if and onlya= candb= d. 19.13 Consideramulti-statemodelforthethreelives( x),(y),(z)wherewehaveeightstates described below, where the stipulated lives are those still living, State0:all;State1:(x ),(y)only;State2:( x),(z)only;State3:( y),(z)only;State4: (x) only; State 5: (y) only; State 6: (z) only; State 7: None. You are given that𝜇01 = 0.02, 𝜇02 = 0.01, 𝜇03 = 0.04, 𝜇34 = 0.02, 𝜇35 = 0.06, 𝜇36 = 0.05,𝜇67 = 0.10 and the force of interest is a constant 0.05. (a) Find the actuarial present value of an insurance policy on the three lives which pays1atthemomentofdeathof(z)providedthat(x )diesfirstand(y)diessecond. (b) Find the probability that the lives die in the order (x),(y),(z). 19.14 A contract is based on a three-state chain. There are benefits at the end of the year of transfer, of 100 for transfer from state 0 to state 2, and 50 for transfer from state 1 to state 2. Annual premiums of 10 are paid while the process is in state 0. There are annuity payments of 5 made each year when the process is in state 1. You are given thati 5 = 0.10 and the transition matrix P5 = ⎛ ⎜ ⎜ ⎝ 0.60 .10 .3 0.20 .70 .1 0.200 .8 ⎞ ⎟ ⎟ ⎠ . If 5V(0) = 50, 5V(1) = 35 5V(2) = 70, find 6V(i),f o ri= 0,1,2. 19.15 Redo Example 19.16 with the following changes in the given data: (i) The transition matrix for the year running from time 5 to time 6 is now given by the matrixP1 of Example 19.1. 332 MULTI-STATE MODELS (ii)Thereisapaymentattheendoftheyearof20000whenahealthylifebecomes unhealthy, and the yearly income for unhealthy lives is reduced to 15 000. 19.16 A insurance on two independent lives (x) and (y) provides for a death benefit at the moment of the second death. The death benefit ise0.05t for death at timet.P r e m i u m s are payable until the second death, The premium is level while both are alive and reduces to one half of the initial amount upon the first death. The life (x) is subject to aconstantforceofmortalityof0.10and( y)issubjecttoaconstantforceofmortality of 0.15. The force of interest is a constant 0.10. Refer to the two-life model as given Figure 19.1. (a) Using the method in Chapter 10, calculate for each of the first three states the annual premium and the reserve at timet. (b) Write down the Thiele differential equations. (c) Verify that your answers to (a) satisfy the equations in (b). 19.17 In a certain four state chain, the forces of transition are given by 𝜇 01(t) = 0.1+0.02t, 𝜇12 = 0.2+0.01t, 𝜇03(t) = 0.1+0.03t, 𝜇13 = 0.2+0.02t. For all otheri,jwithi≠ j, 𝜇ij(t) = 0. The force of interest is a constant 0.06. The following double integral represents the net single premium for a certain insurance contract. ∫ 3 0 ∫ t 0 e−(0.6+0.09t+0.02s)(0.1+0.02s)(0.2+0.01t)dsdt. Describe the benefits on the contract. 19.18 Prove Theorem 19.4 by observing first that pii(s,t+h) = pii(s,t) pii(t,t+h). 20 Introduction to the Mathematics of Financial Markets 20.1 Introduction This chapter will introduce some basic concepts of modern mathematical finance. One goal is to cover the fundamentals of option pricing. This has become an important tool in actu- arial mathematics, since many insurance and annuity contracts today contain the so-called ‘embedded options’, which we discussed in Section 13.2. For the most part, we carry this out in a discrete setting, but we do move into the continuous-time approach briefly in order to introduce the Black–Scholes–Merton formula. Another major objective in this chapter is to revisit the basic quantity of a discount function which we introduced early on. In the first part of the book, we treated this as a deterministic function, but a more realistic approach wouldbetoconsider v(s,t)asarandomvariable,reflectingthestochasticnatureofinvestment returns that we discussed in Chapter 14. In particular, we seek a version of the key identity, Formula (2.1) in this stochastic setting. A prerequisite for this chapter is the starred Section 2.12. We assume familiarity with concepts discussed in that section such as as short selling, forward contracts, and arbitrage. 20.2 Modelling prices in financial markets A financialmarket is an institution designed to facilitate the trading of financial assets, such asstocksorbonds.Certainindividualswishtobuysuchassets,whileotherswishtosellthem, and the financial market provides a forum to bring together the various parties. The potential holderofanassetquotesadesiredsellingprice,knownasthe askingprice,whilethepotential buyer quotes a desired buying price, known as thebidding price. When bidding and asking prices are equal it establishes a price for the asset, and a sale can be made. Our goal in this section is to develop a stochastic model for the evolution of prices. FundamentalsofActuarialMathematics , Third Edition. S. David Promislow. © 2015 John Wiley & Sons, Ltd. Published 2015 by John Wiley & Sons, Ltd. Companion website: http://www.wiley.com/go/actuarial 334 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS For most of this chapter, we assume a relatively simple framework, which will allow us to present the main ideas without too many technical mathematical difficulties. We assume a discrete-time model. That is, trading of assets will take place at integer times 0,1,2,…. The timeperiodcanbearbitrary,andwecanthinkofitaspossiblyaveryshorttime,(sayanhouror evenaminute)whichcouldthenconstituteanapproximationtothemorerealisticcontinuous- time setting. We adopt a finite-time horizon, with timeNas the last date we are interested in. Finally, we assume that the price of any asset can take on only finitely many possible values. Suppose we haveM+1 assets traded in our market, numbered from 0 toM. We will let S j(n) = the price of thej-th asset at timen. Weconsidereachsuchpriceasarandomvariable.Therefore,ourfinancialmarketismodelled byM+1 discrete time stochastic processesS j(n), wherej= 0,1,2, …M, andn= 0,1, …N. Wewillsingleoutaparticularasset,oftenreferredtoasa bankaccount,forassetnumbered 0.Todescribethis,wewillfirstneedtopostulateinourmodelanonnegativequantity rcalled the risk-free rate of interest, which is the interest rate that we can obtain on a risk-free investment as described in Section 2.12. We can then define this asset by S 0(0) = 1, S0(n) = (1+r)n. Inotherwords,thisisanassetwhichaccumulatesattherisk-freeinterestrate.Itisastochastic processinwhicheachrandomvariabletakesasinglevaluewithprobability1.Forsimplicity, we are at first adopting a constant risk-free interest rate. The definition could be based on a more general discount function and we comment on this below in Section 20.13. We will make the same idealized assumptions that we made in Section 2.12. That is, we postulate that for each asset, any real number of the units can be bought at any trading date. Throughshortsellingifnecessary,thisincludesnegativequantities.Wealsoassumethatthere are no transaction costs such as commissions. Notethattheexistenceofthebankaccountmeansthatweareassumingthatallparticipants in our market can freely borrow at the risk-free rate. Another simplifying assumption made throughout is that none of our assets provide any payments at intermediate dates, such as dividends on stocks or coupons on bonds. They provide funds only upon sale or maturity. 20.3 Arbitrage An initial observation is that in the typical financial market, the various asset prices do not move independently. If asseti moves up in price, assetj may have a tendency to move up, or possibly to move down, or be certain to move up or down. It can be quite complicated to model all dependencies, but theno-arbitrageprinciple will often enable us to reduce the possibilities. In our stochastic models, this requires a more complicated definition than the one we gave in Section 2.12. We first define the concept of atradingstrategy. This is roughly a description you, as an investor in the market, would give to an assistant before leaving for a holiday on a remote desert island where you cannot be reached. You would specify the number of units of each ARBITRAGE 335 assettobeheldateachtimeperiod.Ateachtradingdateaftertheinitialportfolioisestablished at time 0, certain assets in the existing portfolio would be sold and others bought to achieve the stipulated amounts. These amounts could depend on the entire past and present history. The description could be enormously complicated or quite simple. For example, a trading strategy might be as follows: Start with an initial portfolio of 1 unit each of asset 1 and asset 2. Keep this intact until the first time that the price of asset 1 is above 40 per unit and the price of asset 2 is below 30 per unit. At that time, sell all units of asset 1 and use the proceeds to buy shares of asset 2. These are then held without further trading. Note that the amounts to be held at timen can depend on all the prices of all assets at or before timen, but not after. It would not be a feasible trading strategy to specify that a a certainassetshouldbesoldattime2,ifthepriceofsomeotherassetwerebelow40attime3. Toformalizethissomewhat,wecanrepresenttheassetholdingsatanytime nbyavector 𝛼(n) = (𝛼 0(n),𝛼1(n)…𝛼M(n)) where𝛼j(n) is the number of units of assetjheld at timen. The entries of this vector are random, depending on the prices up to timen. So a trading strategy is formally a vector of these random vectors.  = (𝛼(0),𝛼(1)…𝛼(N−1)), where each𝛼(r) is a function of the values ofSj(k)f o rj= 0,1, …M, k= 0,1,2 …r. For any trading strategy and any timen, we will have a portfolio consisting of a certain number of units of each of ourM+1 assets. The portfolio at timen will then then have a valueV(n)obtainedbymultiplyingthenumberofunitsofeachassetbythepriceofthatasset at timen, and summing. That is V(n) = M∑ j=0 𝛼j(n)Sj(n), a random variable depending on all prices as well as the trading strategy as followed up to time n. Of courseV(0) is a definite number as it is the cost of setting up the initial portfolio at time 0 when all prices are known. For any trading strategy, there is a reverse strategy which involves holding at each time, the negative of the number of units held in the original strategy. In other words, one sells in placeofbuyingandbuysinplaceofselling.Formally,ifatradingstrategyisgivenbythethe vector , the reverse strategy is given by− .I fV ∗ denotes values for the reverse strategy it is clear thatV∗(n) =− V(n) for alln. Here is another important concept. Definition20.1 Atradingstrategyissaidtobe self-financingifforanytradingdateaftertime 0andbeforetime N,thetotalpriceofalltheassetssoldonagiventradingdateexactlyequals the total price of all the assets bought on that date, so no additional infusion or withdrawal of capital is required. For a self-financing strategy, the value of the portfolio at any intermediate trading date is the same before and after trading. 336 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS Wecannowsummarizetheprocedurewewillbefollowinginsubsequentdiscussions.We set up an initial portfolio at time 0 for a cost ofV(0), dictate a self-financing trading strategy, retreat to the dessert island, where no additional outlays of cash are required and none are received.Finally,theportfolioisliquidatedattime Nforproceedsof V(N),arandomvariable which depends on both the trading strategy and the evolution of prices. We letPdenote the probability measure forV(N). Thekeydefinitionofthissectioncannowbegivenintermsofthestartingvalue V(0)and the ending valueV(N). Definition20.2 Thefinancialmarketadmits arbitrageifthereexistsaself-financingtrading strategy such that V(0) = 0, V(N) ≥ 0, and P[V(N) > 0] > 0. A financial market which does not admit arbitrage is said to bearbitrage-free. In other words, an arbitrage opportunity is one where starting with a zero investment, we cannot possibly lose by the end of the trading period, and we have at least some chance of makingagain.Notethatthearbitrageopportunitydoesnotguaranteeapositivegain.Onecan think of it as being given a lottery ticket for free. We cannot lose anything, and there is some chance of profiting. It is important to note that cases where there is a very small probability of loss do not constitute an arbitrage under this definition. The avoidance of the loss must be absolutely certain. Remark In the definition of arbitrage, we could replace the condition onV(N)b yV(N) ≤ 0 and P[V(N) < 0] > 0, since, if this holds, the reverse strategy will satisfy the original condition.Thislooksabitstrangeatfirst,butitsimplysaysthatifthereisastrategyforwhich we are sure not to gain, then the reverse strategy is sure not to lose. An important consequence of the above is the following. Theorem 20.1 In an arbitrage-free financial market, if there is a self-financing trading strategyforwhichV (n) isaconstantcforsomen,then c= V(0)(1+r) n. Proof. Modify the strategy by holding−V(0) units of the bank account at time 0, so that the new strategy has initial value 0. If necessary, modify the strategy further to stipulate that everythingshouldbesettledattime n,andtheproceeds(possiblynegative)lefttoaccumulate in the bank account at the risk-free rate until timeN. The new strategy will have the constant value ofc−V(0)(1+r) n at timen, and this must be equal to 0. If not, there would be a sure chance of having either a positive or negative amount at timeN, which would imply an arbitrage opportunity, by Definition 20.2 and the remark following this definition. Thiswasareasonablysimpleresult,butthereisanimportantmessagebehindit.Itsaysthat intheabsenceofarbitrage,ifwecanfindaself-financingtradingstrategywhicheliminatesrisk atsomepoint,thenourinitialinvestmentmustaccumulateattherisk-freerateuptothatpoint. OPTION CONTRACTS 337 Remark It is true that any trading strategy can be converted into a self-financing one by using the bank account. An excess of the sales over purchases can be placed in the bank account, while excesses of purchases over sales can be handled by borrowing. However this gives a different strategy with a different amount held in the bank account at timeN, and therefore a different value ofV(N). The self-financing hypothesis is therefore essential in the definition of arbitrage. For our first example, we consider a very simple financial market. We will takeN = 1, so atradingstrategyinvolvessimplyspecifyingtheinitialportfolio.Ourfinancialmarkethas,in addition toS 0, a single risky assetS1 consisting of a stock. We can assume, changing units if necessary,thatthepriceofaunitofthestockis1attime0.Supposethatthepriceofthestock at time 1 can only take two possible values,u or d (standing for ‘up’, ‘down’ respectively) with d< u, each with positive probability. We call this abinomial model to reflect the two possible values at time 1. Theorem 20.2 Theabovefinancialmarketisarbitragefreeifandonlyif d< (1+r) < u. (20.1) Proof. Consider any trading strategy withV(0) = 0. If𝛼 is the number of units of stock in the initial portfolio. we must have−𝛼 units of the bank account. Then we will have either V(1) = 𝛼u−𝛼(1+r)o rV(1) = 𝛼d−𝛼(1+r). Suppose (20.1) holds. If𝛼< 0 then the first such value will be negative and the second will be positive, while the reverse holds if𝛼> 0. If𝛼 = 0, both values are 0. An arbitrage opportunity cannot exist. Conversely if (20.1) is not true, then at least one of two possibilities holds. Suppose d≥ (1+r). We create an arbitrage opportunity by choosing𝛼> 0 which makes both values of V(1) nonnegative,with at least one positive. The other possibility is thatu≤ (1+r), in which case we similarly create an arbitrage opportunity by taking𝛼< 0. Note that the converse statement is intuitively obvious. If the inequality does not hold, then we can create an arbitrage opportunity by either buying a stock which is sure to yield more than the risk-free return, or short selling a stock which is sure to yield less than the risk-free return. Anotherpertinentfacttonoticeinthedefinitionofarbitrageisthattheconditiondoesnot dependontheparticularvaluesof Pbutonlyonwhethersuchvaluesarepositiveorzero.We are therefore led to make use of the following standard definition of probability theory. Definition 20.3 Two probability measuresP and Q on a sample spaceS are said to be equivalent if for allA⊆ S,w eh a v eP(A) = 0 if and only ifQ(A) = 0. (For readers familiar with the concept of an equivalence relation, one can readily verify that this is a legitimate such relation.) It is clear from the definition that a financial market is arbitrage-free with respect toPif and only if it is arbitrage-free with respect to any equivalent probability measureQ. 20.4 Option contracts Given a financial market, we can do more than just buy or sell the existing assets. We have already seen one possibility, which is to enter into forward contracts. Another possibility 338 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS is option contracts.These are in one sense similar to forward contracts since they are both transactions which involve trading of assets at a future date for prices that are specified now. There are major differences however, for in an option, unlike the forward contract, one party isnotobligatedtocompletethetransaction,buthastheoptiontodoso,andwillonlyexercise thisoptionifitisadvantageous.Therearetwobasictypesofoptioncontracts,knownas calls and puts. A buyer of a call option has the right to buy a specified asset at a specified future time, known as theexpirationdate orexercisedate, for a specified price, known as thestrike price or exercise price, if they should choose to do so. The call option buyer has a similar motivation to a speculator taking a long position in a forward contract. They hope for a rise in price, so that they can buy the asset at a price which is lower than prevailing at the time of purchase. If the price of the asset at the expiration date is below the strike price, the option will not be exercised. A buyer of a put option has the right to sell a specified asset on the expiration date for a specified strike price. The put option buyer has a similar motivation to the speculator taking a short position in a forward contract. They hope for a fall in price so thattheycanselltheassetformorethanitisworthatthetimeofsale.Inthiscase,iftheprice of the asset at the expiration date is above the strike price, the option will not be exercised. Unlike the forward contract, the call and put buyers are not on opposite sides. For each of them, there must be another party who sells or (as it is commonly said) writes the option, and agrees to complete the transaction should the option holder so elect. Now if the option is exercised,theoptionwriterisnecessarilysellingorbuyingatanunfavourableprice,andthey are compensated for this by the option price which they receive from the option holder at the time the agreement is entered into. The option writers of course hope that options will not be exercised, so they profit by the full amount of the option price, and do not have to engage in an unfavourable transaction. Determining option prices is complicated, and will form much of the material of this chapter. It should be noted that what we have described are more properly known asEuropean options, which specify that the option can only be exercised on the one specific expiration date. We will assume all options we discuss are of this nature unless specified otherwise. Another type of contract, known as anAmericanoption, allows for the exercise of the option at any time before or on the expiration date. These are more complicated and will be dealt with briefly in Section 20.7. Althoughtheunderlyingassetsforcallsandputarenormallytakenasfinancialinstruments like stocks, as will be the case in our treatment, the basic idea of an option arises in many diversecontexts.Forexample,buyinginsuranceonanassetlikeahouse,isessentiallybuying a type of put option. You are protecting yourself from a drop in value, not from market variationinthiscase,butratherfromphysicaldamage.Similarly,theguaranteesforvariables annuities (as discussed in Section 13.2) which protect your account against unfavourable investmentexperience constituteputoptions.Foranotherexample, supposethatyoutakeout a long-term loan or mortgage, and the lender gives you the right to repay in full at any time without penalty. In effect, you have been given a call option. In this case, you are protected from a rise in the cost of repayment, which will occur if interest rates decline. (Refer to the discussion in Section 2.10.3.) Inessence,protectionagainstdeclinesinthevalueofanassetthatyouown,whileallowing youthefullbenefitofincreasesinvalue,canbeviewedasbeinggivenaputoption.Protection against increases in the value of an asset that you may wish to acquire in the future, while allowingyouthefullbenefitofdecreasesinvalue,canbeviewedasbeinggivenacalloption. Note that in contrast, forward contracts protect you from unfavourable declines or increases, OPTION PRICES IN THE ONE-PERIOD BINOMIAL MODEL 339 but they do not allow the parties to reap the full upside benefits, since the transactions must be completed with the agreed upon prices. 20.5 Option prices in the one-period binomial model In this section, we show how the no-arbitrage principle allows us to calculate an option price for the one-period binomial market where condition (20.1) holds. We illustrate with a particularexample.Supposethatashareofthestockissellingfor108attime0andattime1it willbeeither132or99,eachwithpositiveprobability.Assumearisk-freeinterestrateof10%. Consider a call option on the stock with an expiration date of time 1 and a strike price of 110. What should the price per unit of this option be? At first, one may think that there is no way to determine this exactly, and that it could take on many possible values. After all, the option is just another asset with its price being determined by the amounts bid and asked by the various market participants. The worth of this asset, however, is directly tied to the performance of the stock, so it should be clear that its price must be related in some way to the stock price. Such an asset is often termed aderivativesecurity, since its value is derived from that of another security. Tohelpdeterminetheprice,wetakethefollowingpointofview.Purchasersofcalloptions arenotnormallyinterestedinactuallytakingpossessionofthestockatmaturity.Theysimply wanttobuyitatthestrikeprice,andsellitimmediatelyforthehighermarketpriceifavailable. If the market price is below the strike price, the option is worthless and they receive nothing. The option then is just another assetS 2 with S2(1) = 132−110 = 22 if the stock price goes up, orS2(1) = 0 if the stock price goes down. The problem is to determineS2(0). Thosewell-versedintheactuarialmodelswediscussedinearlierchaptersmaywellthink that we can determineS2(0) by simply taking a discounted expected value, as we did with several other similar sounding problems. That is, we simply take the price as 22vpwherevis thediscountfactorforoneperiod,and pistheprobabilitythatthestockgoesup.Wewillfirst illustrate why one cannot solve the problem this way, and after that, we will, paradoxically, illustrate why onecando it this way. The first problem is that one is not givenp as part of the model. All that we postulated abouttheprobabilitymeasure Pwasthatbothofthepossibleoutcomesattime1havepositive probability.Indeed,theremaynotbeanyreasonablechoiceforasinglevalueof p.Theman y different participants in the market may well have completely different assessments of this figure. It is not unusual to find two experts commenting on a particular stock, where one claimsitisthebestbuyingopportunitytocomealonginthelastdecade,andtheotherpredicts imminent bankruptcy of the firm. The second problem is that one is not givenv. Now the reader may take issue with this statementsincewepostulatedariskfreerateof10%afewparagraphsback,soitappearsasif v issimply(1 .10) −1.Useofthisratewouldimplythatthebuyerislookingforanexpectedreturn of 10%on their investment. However, 10% is the return for a perfectlyrisk-freeinvestment. Investing in a call option is far from being risk-free. If the stock price at expiry is below the strike price, the entire investment is lost. It is to be expected that a rational option purchaser will want a return in excess of 10% as compensation for taking on the risk. (Recall that we discussed the same concept when introducing the risk discount rate in Section 12.4). We will now solve the puzzle, and show that regardless of the assessment ofpor of the desired yields of different individuals, the price of this option can only be 12. The reason is 340 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS that one can in factreplicatethe option for an initial investment of 12. That is, by investing only in the bank account and the stock, one can produce an outcome at the expiration date, which matches exactly the payouts of the option. This is done by buying 2∕3o fas h a r eo f stock at time 0, which will cost 72. We can put in 12 cash, and borrow the additional 60. If the stock price is 132 per share at time 1, we sell our 2∕3 of a share for 88, pay off the loan balance which is now 66, leaving us with 22. If the stock price is 99, we sell our 2∕3o fa share for 66, and pay off the loan, leaving us with nothing extra. We have therefore exactly replicated the option for the price of 12. It is is clear that no one would pay more than 12 to buy this option. Similarly, nobody would sell the option for a price of less than 12, since instead they could reverse the above strategy and be in the same position at time 1 as if they had written the option, but they would have have received 12 at time 0. Here is another point of view, which ties in with our previous definition of arbitrage. If we enlarge our financial market by adding the option as another assetS 2, then we must takeS2 = 12tomakethisenlargedmarketarbitrage-free.Totakeadefiniteexample,suppose the option price is 13. We will construct an arbitrage opportunity. Take the trading strategy which has as initial portfolio𝛼(0) = (−59,2∕3, −1). The reader can verify thatV(0) = 0. Now S 0(1) =− 64.90, so IfS1(1) = 132, thenS2(1) = 22 andV(1) = 1.1. IfS1(1) = 99 then S2(1) = 0 and againV(1) = 1.1. We leave it to the reader to find an arbitrage opportunity if the option price is below 12. Let us now go back to the proposed solution of of 22vpas an option price, which we criticized a few paragraphs above. If we in fact use the risk-free rate and therefore take v= 1.10 −1, we will get the correct answer by usingp= 0.6. Is there someway we could have discovered this probability of 0.6 beforehand? The answer is yes. Let us suppose that thereexistsasocalled risk-neutralindividual,thatisonewhoignorestheriskandishappyto acceptanexpected10%returnonanyinvestment,regardlessofthedegreeofsafetyinvolved. Letpbetheparticularprobabilityofriseinthestockprice,whichwouldbeassumedbysuch a risk-neutral person. In order that this person would be willing to pay 108 for a share of stock, we should have that 108 = 1.10 −1[132p+99(1−p)] and solving we have indeed thatp= 0.6. Wehavenowdiscoveredtheimportantprincipleof risk-neutralvaluation.Theassignment here of 0.6 and 0.4 to the events of the stock going up or down, respectively, is known as a risk-neutralprobability measure. It is the probability that must be assigned by a risk-neutral individualinordertojustifybuyingthestockatthemarketprice.Notethatwearenotsaying that such a person necessarily exists, and indeed have stressed that most investors would be unlikely to possess such an attitude. We are only saying that if one did exist, the price of the underlying asset would necessarily imply a unique probability assessment for that individual. The principle then says that if we use the risk-free interest rate, along with the risk-neutral probability measure, then we can indeed value options by following the usual actuarial approach of taking a discounted expected value. Note carefully that the risk-neutral probability measure need not be the same or indeed haveanyparticularrelationtotheoriginalmeasure P,otherthanbeingequivalentinthesense defined above. Even if we had specified values forP, these would have had no effect on the resultingoptionprice.Thefactthatonecanrisklesslyreplicatetheoptionmeansthatonlythe risk-neutral probability and the risk-free interest rate need be considered. OPTION PRICES IN THE ONE-PERIOD BINOMIAL MODEL 341 This is a puzzling observation at first, and for those who are still skeptical, we will look into the situation a little further. We mentioned above that the probabilityp was not even specified as part of the model, but let us suppose it is. In fact, suppose that instead of a stock with uncertain returns, we have two lotteries each depending on the same random draw. A ball is drawn randomly from an urn containing two white balls and one red. The payoff from lottery 1 at time 1 is 132 if a white ball is drawn, or 99 if a red is picked. The payoff from lottery 2 at time 1 is 22 if a white ball is drawn or 0 if a red is drawn. So the true underlying value ofpisnowindisputableas2∕3.Ifthepriceforalottery1ticketis108,andwemakethe assumptionthatwecanbuyorsellanyfractionoflottery1tickets,thenthepriceforalottery 2 ticket must be 12, by exactly the same argument as given above, regardless of the known value ofp.Whatdoesthisimplyforpeoplewhoparticipate?Buyersofaticketinlottery1are in effect earning an expected return of [(2∕3)132+(1∕3)99)]∕108−1 = 12.04%. There is a reasonable extra return over the risk-free rate, to compensate for the risk taken on. Buyers of a ticket in lottery 2 are in effect earning an expected return of [(2∕3)22∕12]−1 = 22.22%, a much higher return, which compensates for the greater risk in lottery 2 when the entire stake couldbelost.Indeedforanyvalueof pabove0.6,therewillbeareturnabovetherisk-freerate inlottery1andanevenhigherreturninlottery2.Itisonlyfortherisk-neutralvalueof pequal to 0.6, for which the expected returns on both lotteries will coincide with the risk-free rate. Going back to our original example with the stock, is it possible that an investor who assesses the probability of an upward movement as being less than 0.6 would still pay 108 per share, thereby earning an expected return of less than the risk-free rate? This may seem irrational, but it is no more so than the behaviour of a vast number of people who buy lottery tickets or gamble in casinos at highly unfavourable odds. (For more on this topic, see Example 22.2.) Wenextderiveageneralformula.Supposethat(20.1)holds.Aswedidabove,wecanset up an equation to solve forp, the risk-neutral probability that the upward move will occur. TakingS 1(0) = 1, this is 1 = (1+r)−1[pu+(1−p)d], (20.2) which we solve to obtain p= (1+r)−d u−d ,1 −p= u−(1+r) u−d . (20.3) Note that condition (20.1) ensures that 0< p< 1. The above procedure allows us to uniquely price, not only call options, but a general derivative security in this market, which pays an amountA if an upward move occurs orB if a downward move occurs. We do this in one of two ways: first, we can find a replicating initial portfolio consisting of𝛼 units of the stock and𝛽 units of the bank account by solving the equations 𝛼S 1(0)u+𝛽(1+r) = A, 𝛼S1(0)d+𝛽(1+r) = B. (20.4) Then Price = 𝛼S1(0)+𝛽, (20.5) which is the cost of establishing the replicating portfolio. Secondly, and usually easier, we can bypass finding the replicating portfolio and just take the price as the discounted expected 342 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS value of the the payoff with respect to the risk-free interest rate and risk-neutral measure. That is, Price = (1+r) −1[pA+(1−p)B], (20.6) where p is as given in formula (20.3). The reader can verify that both methods lead to the same answer. Example 20.1 For the example given at the beginning of this section, find the price of a put option with a strike price of 110. Solution. If the upward move occurs, the holder tears up the option. If the downward move occurs,theholderbuysthestockfor99,andsellsitfor110.Sothisisaderivativesecuritywith A= 0,B= 11.DirectlyfromEquation(20.6),wehavethatthepriceis1.10 −1(0.4×11) = 4. Alternatively, solve (20.4) to derive the replicating portfolio given by𝛼 =− 1∕3,𝛽 = 40, and use (20.5) to get the same answer. To see this directly, we replicate the option for a cost of 4 byselling1∕3ofashareshort,receiving36,lettingthetotalof40accumulateto44attime1. This allows one to just cover the short position if the stock is up, or cover the short position and have 11 left over if the stock is down. There is in this case yet another way to obtain the answer. In fact, we develop a general formula relating puts and calls. Theorem 20.3 (Put-call Parity) Let 𝛾 denote the cost of a call option and𝜋 denote the cost of a put option on the same stock with a current price of S(0), the same strike price of K,andsameexpirationdateN.Then S(0)+𝜋 −𝛾 = K(1+r) −N. (20.7) Proof. Suppose an investor at time 0 adopts the following trading strategy. Buy one unit of stock, sell one call option, buy one put option, and hold these without further trading up to theexpiration dateN.If S(N) > K,theput willexpire worthless,thecallwillbeexercised by the other party, so that the investor must give up the stock for a price ofK.I fS(N) < K,t h e callwillexpireworthless,theinvestorwillexercisetheputandsellthestockforapriceof K, while ifS(N) = K, both options are worthless and the value is just the stock price. Whatever happens, the value at time N of the portfolio will beK. SinceV(0) is just the left side of Equation (20.7), the formula follows from Theorem 20.1. The proof shows in fact that this theorem is true for a general arbitrage-free market, and does not depend on the binomial assumption. In our present example, we know𝛾 = 12, N = 1, S(0) = 108 andK = 110, and we can immediately calculate that𝜋 = 4. 20.6 The multi-period binomial model The model of the last section is clearly too simple to be representative of reality. As a further extension, we keep the binomial feature, but allow the prices to evolve over several periods. THE MULTI-PERIOD BINOMIAL MODEL 343 We assume that the price of the stock evolves each period as described in the one-period model above. That is,ifthevalue issat thebeginning of aperiod, the priceat theend willbe eithersuorsdwhered< u.Soforaninitialpriceof S(0)attime0,thepriceattime1willbe either S(0)uor S(0)d, and the price at time 2 will be eitherS(0)u2,S(0)ud,o rS(0)d2,e t c .W e can represent this by what is known as abinomialtree. See, for example, Figure 18.1 which is an example withu= 1.2,d = 0.8. We can now consider any general contingent claim, which will be a payoff at timeN which can depend on the entire history of up and down movements in the stock price. To formalize this, consider the sample spaceΩ consisting of all paths in the binomial tree. Each such path can be labeled by anN-termed sequence formed of the entries U and D, where U denotes an upwards branch and D a downwards branch, so there are 2 N paths altogether. We postulate that there is a probability measurePon Ω, but we need not specify anything about it except thatP(𝜔) > 0 for all𝜔 ∈Ω . We now formally define the general type of derivative security we are interested in. Definition 20.4 Acontingentclaim is a contract which provides a payment at timeNwhich is dependent on the particular outcome in our underlying sample space. It is modelled by a random variableX, where for𝜔 ∈Ω ,X(𝜔) is the payment for outcome𝜔. Forexample,acalloptionwithstrikeprice Kandexpirationdate Nonastockwithcurrent price S(0) is a contingent claim given by X(𝜔) = [S(0)u md(N−m) −K]+ whenever𝜔isasequencewith mupwardmovementsand N−mdownwardmovements.(For any real numbert, the symbolt+ denotes max{t,0}.) A contingent claim can be more complicated than the options we have described up to now. Consider, for example, alookback option on a stock which will return at expiry the maximum value of the stock over the period from time 0 to timeN. So looking for example at Figure 18.1, we would have X(DUD) = 96, X(UDD) = 120, and so on. Themulti-periodmodelhasthesameessentialfeaturesthatweobservedintheone-period model. r The financial market consisting of the stock and the bank account is arbitrage-free if and only if condition (20.1) holds. r Any contingent claim can be priced uniquely so as to prevent arbitrage. One method is to find a replicating self-financing trading strategy. The price of the claim is then the cost of setting up the initial portfolio for this strategy. A second way is to take the expected discounted value with respect to therisk-neutral probability measureQ on Ω,whichissimplythemeasureobtainedbyapplyingtheappropriateprobabilities por 1−pas given by formula (20.3) to each branch of the binomial tree. That is, if𝜔 has mentries of U andN−mentries of D, Q(𝜔) = p m(1−p)N−m 344 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS These facts can be verified by using the results from the one-period model, and working backwards in time. The definition of contingent claim gives us directly its value at timeN. We use those to determine the value and strategy applicable to each node at timeN−1, and then use these to get determine the value and strategy applicable to each node at timeN−2, and continue to iterate the procedure until we get to time 0. Example 20.2 Consider a two-period model where the price of a stock evolves as shown by the tree in Figure 18.1 up to time 2, andr= 0.10. The contingent claimXis a call option at time 2, with a strike price of 92. So X(UU) = 52, X(UD) = X(DU) = 4, X(DD) = 0. Find the replicating strategy and the price of the option which will prevent arbitrage, Solution. Supposeattime1,thevalueofthestockis120.Weknowthat u= 1.2,d= 0.8,and we can solve the system (20.4) withA= 52,B= 4t og e t𝛼 = 1,𝛽 =− 920∕11. This means that if the process is in the upper node at time 1, then in order to replicate the payoff at time 2,weshouldown1unitofstock,andcarryadebtof920 ∕11.Thetotalvalue V(1)is400∕11. Similarly,ifthevalueofstockis80attime1,wesolvethesystem(20.4)with A= 4,B= 0, andwearriveatarequiredportfolioof1 ∕8unitsofstockandadebtof80∕11foratotalvalue V( 1 )o f3 0∕11. Wenowmovebacktotime0andagainsolvethesystem(20.4)with A= 400∕11and B= 30∕11,toobtainainitialportfolioconsistingof37 ∕ 44sharesofstockandadebtof7100∕121. The value of this initial portfolio isV(0) = 3075∕121 which must be the price of the option. To summarize, one can replicate this contingent claim by the following self-financing tradingstrategy.Attime0,buy37∕44unitsofthestock,using3075 ∕121ofone’sowncapital andborrowingtheremaining7100∕121attherisk-freerate.Attime1,ifthestockmovesup, increase the stock holding to 1 unit, borrowing additional funds to do so. If the stock goes down, sell enough to reduce the stock holding to 1∕8 unit, using the proceeds to partially repay the loan. This replicating strategy gives in addition ahedgingstrategy. Suppose you have just sold such an option. You run the risk that the stock will move up both periods. If you do not actually own the stock, you will be required to buy it at 144 and sell it at 92 (which shows the danger of selling a so-callednakedoption on a stock you do not own). If you follow the trading procedure outlined above, you will be sure to be able to meet your obligation in any event, assuming of course that the given model for the evolution of the stock price is correct. Tocalculateonlytheoptionprice,ratherthanthecompletereplicatingstrategy,thesecond method can be used. That is option price= (1+r) −N [ ∑ 𝜔∈Ω X(𝜔)Q(𝜔) ] . (20.8) For the particular case of a call option with strike K, this takes the form option price= (1+r)−N [ N∑ m=0 ( N m ) pm(1−p)N−m(S(0)umdN−m −K)+ ) ] , (20.9) THE MULTI-PERIOD BINOMIAL MODEL 345 where p is determined by formula (20.3). In our casep= 3∕4, and we can verify that, as before, the price is (1.1)−2[52(9∕16)+4(6∕16)+0(1∕16)] = 3075∕121 Example 20.3 In the example given above, suppose the interest rate is 0. Find a price and self-financing trading strategy for the so-calledlookback option, which pays at timeN the maximum value of the stock at time 0,1,2. Solution. The payoff is 144 for the outcome UU, 120 for the outcome UD, and 100 for each of the outcomes DU and DD. We can find the price exactly as we did for the option above. Forr= 0 we can calculatep= 1∕2 and Price = [144(1∕4)+120(1∕4)+100(1∕2)] = 116. To find the trading strategy, we need a more complicated diagram. See Figure 20.1. In the original diagram (Figure 18.1), the paths of UD and DU both led to the same position at time 2. This is fine in cases where the prices of the stock at that point was all we were interested in, since this took the same value of 96 on both paths. However, in this case, the contingent claim is path-dependent and we need two different nodes to distinguish the two paths. Fortheuppernodeattime1,weneedtohold 𝛼 unitsofstockand 𝛽 unitsofbankaccount where 144𝛼 +𝛽 = 144, 96𝛼 +𝛽 = 120, so that 𝛼 = 1∕2 𝛽 = 72, V(1) = 132. For the lower node at time 1 we similarly solve 96𝛼 +𝛽 = 100, 64𝛼 +𝛽 = 100. so that 𝛼 = 0, 𝛽 = 100 V(1) = 100. 100 120 80 96 144 64 96 Figure 20.1 Example 20.3 346 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS We are then back to a one period model with an asset that has value 132 if the upward movement occurs, or 100 if the downward movement occurs. So the initial portfolio must have𝛼 units of stock and𝛽 units of the bank account where 120𝛼 +𝛽 = 132, 80𝛼 +𝛽 = 100 so that 𝛼 = 0.8, 𝛽 = 36, V(0) = 116. Sothetradingstrategyistostartwith116(asweknewfromthefirstsolutionabove),buy 0.8 units of stock and put the rest in the bank account. If the upward move occurs, sell 0.3 unitsofstock,orifthedownwardmoveoccurs,sellall0.8unitsofstock,ineachcaseputting the proceeds into the bank account. For contingent claims which depend only on final prices, the first type of diagram, (like Figure 18.1) known as arecombiningtree, provides a significant reduction in computation. This was not readily apparent in our simple example whereN = 2, but suppose instead that N = 10. The recombining tree would have 11 final nodes, while the more general version would have 2 10 = 1024 final nodes. 20.7 American options WeincludeabriefdiscussionhereonAmericanoptions,whichcanhavesomesurprisingand initially puzzling features. Recall that such an option can be exercised at any trading date up to and including the final expiration date. One’s intuition tells us that the price of this should begreaterthanthepriceofthecorrespondingEuropeanoption,sincethereismorechoiceand thereforeachanceformorepotentialgain.Howeverone’sintuitionisnotcompletelycorrect. Infact,givenourassumptions,anAmericancalloptionshouldneverbeexercisedpriortothe expiration date, so in fact the two options are equivalent and should bear the same price. If r= 0, the same phenomenon holds for an American put option. It is never correct to exercise early. However, in the more usual case whenr> 0, it may well be correct to exercise the put option early. We now clarify these rather curious facts. Consider a particular example. You hold an American call option to buy an asset for a strikepriceof100andonacertaintradingdate n< N,theassetpriceis300.Yourdesiretotake advantage of this high price might induce you to exercise the option, making an immediate profit of 200. After all, at a later date the asset price could be lower, with a corresponding reduced gain. However, one should not exercise the option, since there a better way to take advantage of the higher price. You just sell the asset, relying on the option to protect you against further increases, the usual danger with short sales. By doing this and waiting, you would receive 300 immediately. In the worst case scenario, you then have to buy the asset at maturity for 100 and settle your short position. But your gain as of the expiry date would be 200 plus the interest earned on the entire 300 that you received at timen, in addition to an extra gain if the price is below 100 on the expiry date. If you exercised the option early, your gain at expiry would be limited to 200 plus the interest earned on the 200 received at timen. Note that this conclusion depends heavily on our reasonable assumption thatr> 0 and is not true for negative interest rates. AMERICAN OPTIONS 347 Our argument also does not apply to dividend-bearing stocks, since it is possible that by exercisingearlyandreceivingthestock,thedividendspaidwillmorethancompensateforthe loss of interest. It does show however that the only possible times when one should exercise are those which coincide with dividend payments. Exercising before such a date will at least incur the loss of interest up to the dividend payment date. We will not go into the complete analysis in this case. Now consider an American put option. Suppose now that at timenthe strike price is 300 and the price of the asset is 100. Should one take advantage of the low price, by buying at 100 (assuming you don not already own the asset), then exercising the put to sell at 300 and makinganimmediateprofitof200?Iftheinterestrateis0,theanswerisno,becausesimilarly to the call option case there is a better way to take advantage of the current low price. In this reversed situation, we simply borrow 100 and buy the asset, relying on the put to protect us again future drops in the price. At expiry, we sell the asset for a minimum of 300, repay the loan, and have a gain of at least 200. So again with an interest rate of 0, the American and European puts are equivalent. But consider the more realistic case of a positive interest rate. Ourpreviousargumentdoesnotholdnow,sincebywaiting,wearepayingoutinterestrather thanreceivingitasinthecaseofacalloption.Supposethatinanyevent,wedecidetoborrow 100 at timento buy the asset, and the interest charged over the period fromnto N is 5%. If we exercise immediately, we receive 300, which increases with interest to 315 by expiry, and afterrepaymentoftheloan,ourgainattime Nis210.Ifwewaittoexercise,wewillbebetter off if and only the asset price is higher than 315, in which case we keep the asset and tear up the option. So it is not immediately clear whether to exercise or not. In our discrete model, we can effectively work out the price and trading strategy for an American put option by the same backwards induction process that we illustrated in Examples 20.2 and 20.3. One simply must do an extra comparison at each node. Suppose we have calculated data for all nodes at times greater thann and we are considering a node at time n. One first works out the strategy and a temporary valueV exactly as in the European case. One then compares that value with what could be obtained from immediate exercise at time n. This is calculated by buying (or selling) a sufficient quantity of the asset so that you hold one unit and then selling that unit for the strike price. If this exercise value is greater thanV, then that replacesV as the value, and the strategy is to exercise at that node. The following is a simple one-period example, which is sufficient to illustrate the tech- nique, since, in all cases, you just follow the procedure below at each node. Example 20.4 An asset sells now for 100, and at time 1, will have a price of either 120 or 80, both with positive probability. The risk-free interest rate is 0.10. Find as a function ofK the price of an American put option with a strike price ofK. Compare this with the price of a corresponding European put if (i)K = 113 and (ii)K = 102. Solution. The value at time 1 is (K−120) + for an upward move and (K−80)+ for a downward move. By (20.3), the risk-neutral probability of the downward move is 1∕4. In the extreme case thatK ≤ 80, the option is clearly worth nothing. Take the other extreme where K ≥ 120. The value at time 0 in the European case would be [(3∕4)(K−120)+(1∕4)(K− 80)](1.1 −1) = K(1.1)−1 −100 which is less thanK−100. The price of the option isK−100 and the strategy is to exercise immediately at time 0. 348 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS Now consider the case when 80< K < 120. The price will be the maximum of {K−100, 0.25(K −80)(1.1)−1} where thesecond term isthepriceof theEuropean put. Wecan solve toshow that immediate exercise is optimal precisely when K ≥ 1800∕17 = 105.88. So forK = 113 the price of the American put is 13 as compared to 7.50 for the European put. WhenK = 102, the price of both options is 5. 20.8 A general financial market We often wish to model situations which are much more complicated than the ones we consideredintheprevioussection.Foronething,wemayhaveseveralriskyassetsratherthan one. For another, the evolution of prices may be given by a more involved structure than the binomial tree, and even in the binomial case, it may have a more complicated form than the constant up and down ratios ofuandd. Atypicalexampleofsuchageneralmarketismodelledasatreelikeevolutionwhichwill apply to all assets. See for example Figure 20.2 At each timenwe have a number of nodes, which we can think of as representing a certain ‘state of nature’, and all the asset prices are determined by this state. This market has three assets and the prices (S 1(n),S2(n)) are shown at each node. The asset 0 prices need not be shown as they are known completely once we specifyr. We will now describe the general discrete-time model. The notation is necessarily some- what involved, but the market of Figure 20.2 is sufficient to capture the main ideas. We have a finite state stationary Markov chain (see Section 18.2) with the following special structure. The set of states is divided up into subsets 0,1 …N, wherek denotes the set of states at timek. There is a single states0 ∈ 0 reflecting the fact that there is no uncertainty about prices at time 0. The only transitions of positive probability are those from a state to another one which is one period later. That is, given statesiin  k and jin m where mis notk+1, we must havepij = 0. For example, in our multi-period binomial model, there werek+1 states at each timek, and for each state at timek, there were exactly two transitions into a state at timek+1. In the market of Figure 20.2, we would have0 = 0, 1 = {1,2,3}, 2 = {4,5,6,7,8,9,10}. In the binomial case, we described outcomes by sequences consisting ofUand D.I nt h e general case, where we can have more than two branches from a node, we need a somewhat different representation. The evolution of our system up to timek can be described by a sequence of states {s 0,s1,…,sk} where eachsk ∈ k and psj,sj+1 > 0f o rj< k. We will call such a sequence anadmissiblek-sequence. We now take our sample spaceΩto consist of all admissible N-sequences, and this is equipped with the probability measurePwhere for any admissible N sequence 𝜔,w eh a v eP(𝜔)=Π N−1 k=0 psk,sk+1. As an example, in the market of Figure 20.2,Ω consists of the 7 elements {{0,1,4}, {0,1,5},{0,2,6},{0,2,7},{0,3,8},{0,3,9},{0,3,10}}. A GENERAL FINANCIAL MARKET 349 120, 130 State 1 100, 90 State 2 100, 100 State 0 130, 160 State 4 110, 100 State 5 105, 100 State 6 95, 80 State 7 90, 100 State 8 80, 70 State 9 70, 70 State 10 80, 80 State 3 Figure 20.2 A finanical market with two risky assets It is convenient to adopt the following notational device. For any admissiblek-sequence 𝜈 withk< Nwelet v◦ denotethesetofall 𝜔 ∈Ω whichextend 𝜈.Thatisif v= s0,s1,…,sk, then v◦ = {𝜔 ∈Ω: the first k+1 entries of𝜔 are s0,s1,…sk,}. (20.10) For example, in the market of Figure 20.2, {0,3}◦ will denote the subset {{0,3,8},{0,3,9}, {0,3,10}}. 350 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS Wenowwanttocaptureformallythenatureofquantitieslikeassetpricesorassetholdings atacertaintime k.Thesearerandombeforetime k,butarethenknownwithcertaintyattime korlater.Forexample,inthemarketofFigure20.2,thepriceofasset1attime1isarandom variable which is uncertain at time 0, but then is known precisely at time 1. So it will take the constant value of 120 on the set {0,1} ◦ = {{0,1,4},{0,1,5}}. Similarly it will take the constant value of 100 on the set {0,2}◦ and the constant value of 80 on the set {0,3}◦. We handle this in general by the following definition. Definition 20.5 For any integerk= 1,2, …,N, a random variableV defined onΩis said to be k-determinedif V is constant on any set𝜈◦ where𝜈 is an admissiblek−sequence. For ak-determined random variableV, we will writeV(𝜈) to denote the constant value of V on the set𝜈◦. Itfollowsfromthedefinitionthata k-determinedrandomvariableisalso m-determinedif m≥ k. This just reflects the fact that if we know something at timek, we will know it just as well at some later time. Note that a 0-determined random variable is just a constant, since the only 0-admissible sequence is that with the single entry ofs0 ands◦ 0 is the entire setΩ. Definition 20.6 For any random variableW, we define ak-determined random variable Ek(W) as follows. Suppose that the point𝜔 ∈ 𝜈◦. That is,𝜈 comprises the beginningk+1 entries of𝜔. We then define Ek(W)(𝜔) = E(W|𝜈◦) using the notation of (20.10). So Ek(W) then just gives the expected value ofW conditional on the firstk-steps of the evolution. Itisclearfromthedefinitionthat Ek(W)isk -determined.E0(W),being0-determined,isa constant and just equal to the usual expected valueE(W). Consider the other extreme where k= N.Thenforany 𝜔,theset𝜈 ◦ isthesinglepoint 𝜔andE(W|𝜔)isjustW (𝜔),showingthat EN(W) = W.S oa skincreases,Ek(W) gives us more and more information aboutW until we reach timeN and knowW exactly. Example20.5 ConsiderFigure20.2.Supposethatwheretherearetwobranchesemanating from a node, the probability of an upward move is 2∕3 and that of a downword move is 1∕3, whileinthecaseofthreebranchesemanatingfromanode,eachhasprobability1∕3.Describe the random variableE 1(S1(2)). Solution. Consider the setB= {0,1}◦, which consists of the two points, namely {0,1,4} that has probability 4∕9 and {0,1,5} that has probability 2∕9. We could calculate that the conditional probabilities givenB are (4∕9)∕(6∕9) = 2∕3 for {0,1,4} and (2∕9)(6∕9) = 1∕3 for{0,1,5}.Observenowthatwedidnothavetodoallofthiscalculation,sincethetree-like structuremakesitpossibletoreadofftheseconditionalprobabilitiesfromthefuturebranches of the tree without worrying about the past. In this case, with only one future step, they are immediate. We then have thatE 1((S1(2)) takes the value of 2∕3(130)+1∕3(110) = 123 1∕3 on{0,1,4}and{0,1,5}. Similarly,ittakesthevalueof1012∕3on{0,2,6}and{0,2,7}and the value of 80 on {0,3,8},{0,3,9} and {0,3,10}. ARBITRAGE-FREE CONDITION 351 To summarize, a financial market withM+1 risky assets and of durationN is modelled by aMarkov chain withthe special structureas noted above, a probability measure onthe set Ω of all paths from time 0 to timeN, a risk-free interest rater, and random variablesS j(n), j= 0,1, …M, n= 0,1, …,N,o n Ω where each Sj(k)i s k-determined. A trading strategy consists of a collection of random variables𝛼j(n), j= 0,1, …M, n= 0,1, …N−1 where each 𝛼j(k)i sk-determined. For an important application to follow, we now turn to the concept of amartingale introduced in Section 18.3, and look at conditions for this to occur in our present context. Fix a probability measureQonΩwhich is equivalent toPand let {W n},n= 0,1, …,N be a sequence of random variables such that eachWk is k-determined We claim that this will be a martingale, provided Ek(Wk+1) = Wk, k= 0,1, …,N−1. (20.11) To see this, suppose that the above holds. Fix anyk and a sequence of real numbers {w0,w1,…,wk}. Consider any set which has positive probability underQand is of the form A= {𝜔 ∈Ω: Wi(𝜔) = wi,i= 0,1, …,k}. Nowbydefinition,membershipin Aisdeterminedbywhathappensuptotime k.Ifasequence 𝜔 ∈ A, any sequence which has the same firstk+1 entries must also be inA. This implies thatAmustbetheunionofsubsetsoftheform 𝜈◦ forsome k-admissiblesequence 𝜈. Forany such𝜈, it follows from our hypothesis (20.11) that E(Wk+1|𝜈◦) = Ek(Wk+1)(𝜈◦) = Wk(𝜈) = wk and from (A.22), (applied to the sample sample spaceA with the conditional probability Q(⋅|A) we can conclude that E(Wk+1|A) = wk, showing that the sequence is a martingale. To apply this, refer again to Figure 20.2. Let Q be the probability measure which assigns 1∕3toeachtransitionwhentherearethreetransitionsoutofastateand1∕2toeachtransition whentherearetwo.Wecanthenseethatthesequenceofpricesofasset1isamartingaleunder thismeasure,bysimplyverifyingthecondition(20.11)ateachnode.Forexample,atstate3,we havethatthevalueof S 1(1) = 80andthevalueof E1(S1(2)) = (1∕3)90+(1∕3)80+(1∕3)70 = 80. The same holds at all other states. Similarly, we can show that the same holds forS2(n), the sequence of prices of asset 2. 20.9 Arbitrage-free condition To decide when a general financial market is arbitrage-free, directly from Definition 20.2, couldbeextremelycomplicated.Wewouldhavetoconsiderallpossibleinitialportfolioswith value 0 and all possible self-financing trading strategies. Fortunately, there is often a faster way. Suppose we can find a probability measureQ, equivalent toP, such that for each asset i, the sequence {S i(n)} is amartingaleunderQ. Consider any self-financing trading strategy. 352 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS At any timen, the portfolio has a valueV(n). For each asseti, the expected value at time n+1 will again beSi(n) and so the expected value of the portfolio before trading will be V(n). Since our trading strategy is self-financing, the expected value after trading will again be V(n). Since this is true for all possible values of the portfolio at timen,w em u s th a v e that EQ[V(n+1)] = EQ[V(n)]. (The subscript indicates that expectations are with respect to the probability measureQ.) Working inductively, we have thatEQ[V(N)] = V(0) = 0. It is impossible forV(N) to be nonnegative for all outcomes, have a positive probability of being positive, and still have an expectation of 0, so we cannot have an arbitrage opportunity. This seems like a nice simple answer but on the face of it there is a major problem. It is not reasonable to expect that our stochastic processes for stock prices are martingales, as we indicated in Section 18.3. In fact, the bank account, by definition, cannot be a martingale unlessr= 0.Soourresultabovemayappearatfirsttobemeaningless,butthefollowingtrick saves the day. We do not have to measure our assets in terms of dollars. They can be expressed relative to some other asset. Define ̂S j(n) = Sj(n)∕S0(n). That is,̂Sj(n)i st h ev a l u eo fa s s e tjat timenin terms of the bank account. We can think of ̂Sj(n)asadiscounted orpresentvalue,sinceitiswhatwewouldhavetoinvestinourrisk-free bank account in order to accumulate toSj(n) at timen. It is a random variable rather than a number sinceSj(n) is a random variable. The same argument we gave above clearly goes through if eacĥSj(n) is a martingale. This is now possible sincêS0(n) takes a constant value of 1. We have therefore proved the ‘if’ direction of the following major result. Theorem 20.4 (The fundamental theorem of asset pricing) A financial market is arbitrage-free if and only if there is a probability measure Q onΩ which is equivalent to P,andforwhich ̂S j isamartingaleforj = 1,2, …,M. A major example is the the multi-period binomial model, where the given risk-neutral measure satisfies the conditions of the above theorem, as we verify from Equation (20.11). Indeed, suppose that̂S(k) = s, so thatS(k) = s(1+r) k. Referring to Equation (20.3), for any 𝜔 ∈Ω , Ek(Sk+1)(𝜔) = s(1+r)k u[(1+r)−d)+d[u−(1+r)] u−d = s(1+r)k+1 so that Ek[̂S(k+1)] = s. As another application, we can conclude immediately from our observations in the pre- ceding section that the market of Figure 20.2 is arbitrage-free whenr= 0. Note that the risk-neutral probabilityp that we gave in the one-period binomial market was the only possible value that would makêS1 a martingale, as shown by Equation (20.2). The terminology is carried over and any probability measureQsatisfying the conditions of Theorem 20.4 is known as arisk-neutralmeasure. The main conclusion of this section then is that usually the best way to show a given a given financial market is arbitrage-free is to show the existence of a risk-neutral measure. The converse of the fundamental theorem will be proved in the following section. EXISTENCE AND UNIQUENESS OF RISK-NEUTRAL MEASURES 353 20.10 Existence and uniqueness of risk-neutral measures 20.10.1 Linear algebra background To complete our study of financial markets, we require a knowledge of some facts in linear algebra. We assume familiarity with the concept of a linear space (also known as a vector space) and linear subspaces. We also assume familiarity with the concepts of closed and bounded sets. Any basic text on multivariate calculus should contain the necessary details. The following is a brief review, adapted to our ultimate goals. Consider in particular the vector spaceW consisting of all real-valued functions defined on some finite setS, with the operations of point-wise addition and scalar multiplication. This is ann-dimensional space wheren is the number of points inS. We let 0 denote the function which takes the value 0 at each point ofs. (The context should distinguish this from the number 0.) For anyf,ginW, we have an inner product f ⋅ g= ∑ s∈S f(s)g(s). A subsetKof W is said to beconvexif it contains the line segment joining any two of its points.Thatis,given f andginKandandascalar0 <𝛾< 1,thefunction 𝛾f +(1−𝛾)gisin K. AhyperspaceinWisaproperlinearsubspaceofmaximumdimension,thatisonelessthan thedimensionofthespace.Soforexample,ahyperspacecanbevisualizedintwo-dimensional spaceasalinethroughtheorigin,orinthree-dimensionalspaceasaplanethroughtheorigin. We need the following two facts about hyperspaces. The first is a fairly standard result and not difficult to verify. The second is quite a bit more advanced. 1. Anyhyperspace Hisdeterminedbyitssocalled orthogonalvector.Thatis,thereisan elementq≠ 0i nW such that H = {h∈ W : q⋅ h= 0}. The elementq is unique up to a scalar multiple. In two or three dimensions, we can visualize it geometrically as a vector perpendicular toH. 2. LetLbeanylinearsubspaceof Wandlet Kbeaclosedandboundedconvexsetthatdoes notintersect L.Thenthereahyperspace HcontainingLsuchthat Kdoesnotintersect H. It is simple enough to visualize this geometrically in three-dimensional space. If a linedoesnotintersectaclosedandboundedconvexset,wecanfindaplanecontaining the line which does not intersect the set. This of course does not hold if the set is not convex. Suppose thatK is a doughnut-shaped region, and the line goes through the hole. Then any plane containing the line must intersectK. 20.10.2 The space of contingent claims We return now to our model as described above and apply our linear algebra concepts. For a given financial market, define the following sets. Let W be the linear space of all real valued functions onΩ. 354 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS We can view this as the space ofcontingent claims, those payments at timeN which are determined by the particular path. An important subspace ofW is given by L= {f ∈ W : there exists a self-financing trading strategy such that for all𝜔 ∈Ω , f(𝜔) = V(N)(𝜔)}. So L is the subspace of allreplicable claimsas defined above in Section 20.5. It is a linear subspace since, givenf and g in L, we can replicatef +g by just holding at each stage the sumoftheholdingsinthetradingstrategiesreplicating f andg,andwecansimilarlyachieve any scalar multiple off by multiplying our holdings by that scalar. Let L0 = {f ∈ L: there is a replicating self-financing trading strategy forf with V(0) = 0 This easily seems to be a linear subspace ofL.W el e t K = { f ∈ W : f(𝜔) ≥ 0 for all𝜔 ∈Ω , ∑ 𝜔∈Ω f(𝜔) = 1 } . which is a convex subset ofW. So a nice linear algebra definition for a financial market to be arbitrage-free is to simply say thatL0 does not meetK. (Of course any nonzero, nonnegative functionf inL0 represents an arbitrage opportunity, but an appropriate scalar multiple of such anf will be inKand also in the subspaceL0. We also use the fact that our original probability measurePmust take a positive value on each𝜔.) To illustrate, Figure 20.3 gives a geometric picture of the one-period binomial market. Any contingent claim is represented by a point (f(U),f(D)) in the plane. The setKis the line f(D) L0 ‐ arbitrageL0 ‐ arbitrage-free (0,1) K (1,0) f(U) Figure 20.3 A picture of the one period binomial market EXISTENCE AND UNIQUENESS OF RISK-NEUTRAL MEASURES 355 100 120 80 100 Figure 20.4 A market in which not all contingent claims are replicable segmentjoiningthepoints(0,1)and(1,0).Thesubspace L0 isapropersubspaceandtherefore must be a line through the origin. In the arbitrage-free case, this line will have negative slope and not meetK. In the case of an arbitrage opportunity,L 0 as represented by the dotted line, has a slope that is either nonnegative, or equal to∞, and it must intersectK. The picture also makesitclearthat Listheentireplane,aswenoticedenough,sinceitasubspacethatproperly contains L0. Further examples are furnished by the markets of Figures 20.4, 20.5 and 20.6. Assume that r= 0. Alternatively, we can assume any positiverand interpret the asset values that are given as aŝSj(n) rather thanSj(n). The conclusions will be the same in either case. In the single risky asset market of Figure 20.4, the setΩ will have three pointsU,M,D (for‘up’,‘middle’,‘down’).Iftheinitialportfoliohas 𝛼 unitsofstockand 𝛽 unitsofthebank account, the time 1 value of portfolio will be 120𝛼 +𝛽 for the upward movement, 100𝛼 +𝛽 if the price stays the same, or 80𝛼 +𝛽 for the downward movement. It follows that L= {f : f(U)+f(D)−2f(M) = 0}, atwo-dimensionalsubspaceof W,showingthatnotallcontingentclaimsarereplicableinthis market.Inparticular,acalloptionwithstrikeprice110willhave f(U) = 10,f(M) = f(D) = 0 which is not inL. For initial portfolios of value 0, we must have in addition that 100𝛼 +𝛽 = 0 leading to L0 = {f : f(M) = 0:f (U)+f(D) = 0}. 100, 100 130, 120 60, 80 Figure 20.5 A market that is not arbitage-free 356 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS 111, 91 120, 100 90, 70 110, 90 Figure 20.6 See Exercise 20.7 The intersection ofL0 and K is clearly 0, showing that this market is arbitrage-free. Of course we could have immediately deduced this from the Fundamental Theorem, since assigning probabilities of 1∕3 to each branch yields a risk-neutral measure. Consider the two risky asset market of Figure 20.5. An initial portfolio with value 0 will be given by the vector of the form 100(−(𝛼 +𝛽),𝛼,𝛽). Then, a functionf will be inL 0 if we can find𝛼,𝛽 satisfying 30𝛼 +20𝛽 = f(U), −40𝛼 −20𝛽 = f(D). (20.12) These equations are readily solved to give 𝛼 =− (f(U)+f(D))∕10,𝛽 = (4f(U)+ 3f(D)∕20.Itfollowsthat L0 = L= W.Sothismarketisaboutasfarfrombeingarbitrage-free as we could possibly get. Any contingent claim can be replicated for an initial cost of 0! Obviously, the prices here as shown could not be maintained by rational investors. To see the delicacy of situations like this, look at this market again, but make the modifi- cationthat S 1(1) = 70insteadof60.Weleavetothereadertoverifythatwestillhave L= W, but L0 is quite different. The coefficient of𝛼 in the first equation of system (20.12) is 40 instead of 30, so that now L0 = {f ∈ W : f(U)+f(D) = 0 as in the market of Figure 20.3, which shows that the market is arbitrage-free. We can also deduce this immediately from the Fundamental Theorem, since now there is a risk-neutral measure, Q(U) = Q(D) = 1∕2. The market of Figure 20.2 that we investigated in Section 20.8 is more complicated, and it would involve a great deal of calculation to try to deduceL 0 exactly, although we do know thatitcannotmeet Kduetothearbitrage-freecondition.Itispossibletoshowthat L= W,but thisisfarfromobviousfromthefiguresasgiven.Asaparticularcase,considerthefollowing. Example 20.6 Let X be the contingent claim that that takes the value 60 on UU and 0 elsewhere. Taker= 0. Find a self-financing trading strategy to replicate this claim. Solution. For any such strategy, each of the lower two nodes at time 1 will lead to a claim of 0 at time 2, so by the martingale property, our portfolio must have value 0 at these nodes. Therefore, looking at the pre-trading values at time 1, 𝛼 0(0)+100𝛼1(0)+90𝛼2(0) = 0, 𝛼0(0)+80𝛼1(0)+80𝛼2(0) = 0, EXISTENCE AND UNIQUENESS OF RISK-NEUTRAL MEASURES 357 which gives 𝛼1(0) =− 0.5𝛼2(0), 𝛼0(0) =− 40𝛼2(0). Similarly,thevalueattheuppernodeattime1mustbe60timestheprobabilityofanupward move, which is 60(1∕2)= 30. So 𝛼 0(0)+120𝛼1(0)+130𝛼2(0) = 30, and substituting from above we have, 𝛼0(0) =− 40, 𝛼1(0) =− 0.5, 𝛼2(0) = 1. To summarize, the trading strategy at time 0 is to buy 1 unit of asset 2, financing this by selling 1∕2 unit of asset 1, borrowing 40 and putting up the remaining 10. This checks out since the initial cost must be 60Q(UU) = 60(1∕3)(1∕2). At time 1, if the middle or lower branchoccurs,selltheunitofasset2,whichisjustenoughtocovertheshortpositionandpay off the loan. Wemustnowdecidewhattodoattime1iftheupperbranchoccurs.Inthiscase,wehave 𝛼 0(1)+130𝛼1(1)+160𝛼2(1) = 60, 𝛼0(1)+110𝛼1(1)+100𝛼2(1) = 0. Thereareseveralsolutionstotheseequations,whichindicatesthatareplicatingself-financing trading strategy need not be unique. One example is to take 𝛼 0(1) =− 100, 𝛼1(1) = 0, 𝛼2(1) = 1. This strategy involves borrowing an additional 60 to cover the short position in asset 1. At time 2, we pay off the loan of 100, and have either 60 or 0 left, depending on what happened to asset 2 at time 2. Similarly, for each of the other six paths, we could find the replicating strategy for a claim which pays off only on that path. We would take a suitable linear combination of these seven strategies to replicate any possible contingent claim. This will show thatL= W.I n Section 20.11, we will prove a result which provides a much easier way to see this. 20.10.3 The Fundamental theorem of asset pricing completed In this section, we prove the converse to the result established above, and show that in any arbitrage-free financial market, wecan find arisk-neutralmeasureQ. We willproceed intwo stages. Stage 1:Defining Q: Bythearbitrage-freeassumption, L 0 doesnotmeettheset K,aconvex,closedandbounded set. By our linear algebra results of Section 20.10.1 we can find a hyperspaceH containing L0 and not meetingK.L e tqbe an element orthogonal toH. That isH = {h: q⋅ h= 0}. Nowitcannotbethatfortwodistinctpoints f andginK,weha v eq⋅ f < 0and q⋅ g> 0, for if so we could find𝛾 such that the functionh= 𝛾f +(1−𝛾)g satisfies q⋅ h= 0 and so 358 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS h∈ H. But by convexityh∈ K, and this would contradict the fact thatH does not intersect K. So by a change of sign, if necessary, we can assume thatq⋅ f > 0 for allf ∈ K.N o wi n particularthefunctions1 𝜔 whichtak ethev alueof1on𝜔 andthevalue0elsewherearein K, and so we can infer that for all𝜔 ∈Ω , q(𝜔) = q⋅ 1𝜔 > 0, and by multiplying by a suitable scalar we can ensure that ∑ 𝜔∈Ω q(𝜔) = 1, whichmeansthatthefunction qistheprobabilityfunctionforaprobabilitymeasure QonΩ. Stage 2:Showing the martingale condition: Fixany j.Wewillshowthatthestochasticprocess ̂Sj isamartingaleunder Q.Foranytime n< N and any possible values of Sj(n), letA denote the event thatSj(n) = s, which means that ̂Sj(n) = s(1+r)−n. We must show that EQ(̂Sj(n+1)|A) = s(1+r)−n, or equivalently that EQ(Y|A) = 0, (20.13) where Y = ̂Sj(n+1)−s(1+r)−n. Consider the following trading strategy. Do nothing before timen. If the price of assetj at time nis not equal tos, do nothing at all. If the price at that time iss, buy 1 unit of assetj, borrowing to do so, and sell it at timen+1. Apply the proceeds to repaying the loan and let thedifference(whichcouldbenegative)accumulateinthebankaccount.Let f bethefunction inW corresponding to this strategy. If Sj(n) = s, this strategy yields a bank account of [Sj(n+1)−s(1+r)] at timen+1. Multiplying by (1+r)N−n−1, the accumulated amount at timeN is (1+r)NY if the purchase is made and 0 if the purchase is not made. Our trading strategy is self-financing and requires an initial investment of 0. Therefore, the functionggiven by g(𝜔) = { (1+r) nY(𝜔)i f 𝜔 ∈ A 0, if 𝜔 ∉ A is inL0. This means that 0 = q⋅ g= ∑ 𝜔∈A q(𝜔)(1+r)NY(𝜔) = (1+r)NQ(A)EQ(Y|A). The fact thats is a possible value ofSj(n)i m p l i e st h a tQ(A)> 0, and so we must have EQ(Y|A) = 0, establishing Equation (20.13). COMPLETENESS OF MARKETS 359 One method of showing that a market is not arbitrage-free is to find the subspaceL0 and show that it intersectsK. But as we saw above, this can be computationally infeasible in all but the simplest cases. The converse of the Fundamental Theorem provides an easier way. Example 20.7 Use the above result to show that the financial market of Figure 20.5 is not arbitrage-free. Solution. Given a probability measure onΩ,l e tq be the probability of an upward move. For ̂S 1 to be a martingale, we need 130q+60(1−q) = 100 so thatq= 4∕7. For̂S2 to be a martingale, we need that 120q+80(1−q) = 100 so thatq= 1∕2. No such measure exists. 20.11 Completeness of markets Inthissection,weposethefollowingquestions.Givenanarbitragefreemarket,canwepriceall contingentclaimsbythetwomethodswehadinthebinomialmodel?Canwedosouniquely? Theuniquenessquestioniseasilyansweredforanarbitrage-freemarket.Asweshowedin thebinomialcase,ifwecanreplicateacontingentclaimwithaself-financingtradingstrategy, then the cost of that claim should be the costV(0) of setting up the initial portfolio. What happens, however, if there are several different replicating self-financing trading strategies? This can certainly occur, but inan arbitrage-freemarket, they necessarilyhave thesameV(0) whichmeansauniqueprice.Supposetothecontrarythatthereweretworeplicatingstrategies for the same contingent claim, one with an initial cost of 100 and the second with an initial cost of 60. The investor could follow both the second strategy and the reverse of the first strategy for a net gain of 40 at time 0 which would be placed in the bank account, resulting in an overall initial value of 0. At timeN, the payments on these two strategies would cancel, leaving a certain positive amount in the bank account, contrary to the fact that there were no arbitrage opportunities. We turn now to the existence question, beginning with a definition. Definition 20.7 A financial market is said to becompleteif, given any contingent claimX, there is a self-financing trading strategy that replicatesX. In other words, using the notation of the preceding section, the subspaceLis all ofW. We have already shown that the multi-period binomial market is complete. Moreover in Figure 20.4, we gave an example of an incomplete market. Thefollowingtheoremgivesacharacterizationofcompletenessforarbitrage-freemarkets. Theorem 20.5 An arbitrage-free market is complete if and only if there is a unique risk- neutralmeasure. Proof. Suppose that the market is complete. LetQ be any risk-neutral measure. Fix any 𝜔 ∈Ω .L e tX 𝜔 be the contingent claim that pays 1 if𝜔 occurs and pays 0 for all other outcomes, and choose a self-financing trading strategy that replicatesX𝜔.I fV(0) is the cost of the initial portfolio, the martingale property ensures that V(0) = (1+r)−NEQ(X𝜔) = (1+r)−NQ(𝜔), 360 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS so that Q(𝜔) = V(0)(1+r)N, showing thatQis uniquely determined. Conversely,supposethatthemarketisnotcomplete,sothat Lisnotequaltoallof W.W e can then choose a nonzero functionh∈ W such that h⋅ f = 0 for allf ∈ L. (20.14) (Since we can do this for a hyperspace, we can clearly do it for any proper subspace which is contained in some hyperspace.) The function which takes the constant value 1 is inL, (achieved by investing (1+r) −N in the bank account at time 0), so we must have that ∑ 𝜔∈Ω h(𝜔) = 0. (20.15) Let Q be the probability measure with the probability functionq as constructed in proving the ‘only if’ part of Theorem 20.4. We will produce a different functionq′ with the same properties asq, namely q′(𝜔) > 0, for all𝜔 ∈Ω , (20.16) ∑ 𝜔∈Ω q′(𝜔) = 1, (20.17) q′ ⋅ f = 0, for allf ∈ L. (20.18) Then q′ will induce a second martingale measureQ′. To constructq′, we note that since q(𝜔) > 0 for all𝜔, we can choose a positive number𝛿 sufficiently small so that the function q′ = q+𝛿h satisfies (20.16), and sinceh≠ 0, q′ is different fromq. In view of Equations (20.14) and (20.15), it is clear thatq′ also satisfies Equations (20.17) and (20.18). Toillustratetheproofofthelastpart,lookagainatthemarketofFigure20.3.Thefunction q(𝜔) = 1∕3 for all𝜔gives us a risk-neutral measure. The space L is, as we have seen, the set of all functions such that f(U)−2f(M)+f(D) = 0 and the perpendicular functionhcan be taken as h(U) = 1, h(M) =− 2, h(D) = 1. We can take then𝛿 to be any number strictlybetween−1∕3 and 1∕6, which yields an infinite number of risk-neutral measures. As a consequence of the above theorem, we can see immediately that the market of Figure 20.2 is complete, without going through the somewhat involved calculation of the replications that we did before. The probability assignment which we gave in Section 20.8 is clearly the unique risk-neutral measure. THE BLACK–SCHOLES–MERTON FORMULA 361 Incompleteness means that there are not sufficiently many assets to account for all the variations in possible contingent claims. Comparing Figures 20.4 and 20.5, we see that with three branches we need at least three assets in order to achieve completeness. This explains alsothefactthatwithonlyoneriskyasset,weneedabinomialmodeltoachievecompleteness. For an additional example, consider the two risky asset market of Figure 20.5. We leave it to the reader to decide whether or not it is arbitrage-free, and whether or not it is complete. Atthispoint,wesummarizeourconclusions.Supposewehaveanarbitrage-freefinancial market. If the market is complete, then for any contingent claimX, there is a unique price whichwillpreventarbitrageopportunities.Thiscanbefoundinoneoftwoways.First,choose a self-financing trading strategy to replicateX, and the price will beV(0). Second, take the discounted expected value of (1+r) −NX with respect to the unique risk-neutral measureQ. If the market is incomplete, then we can still find a unique price for the replicable contingent claims. However the non-replicable claims cannot be priced uniquely, since different choices ofarisk-neutralmeasurecangivedifferentresults.Someothercriteriamustbeusedtoarrive at prices. This does not mean of course that there are no restrictions on the price of such claims. Often, a range of values can be computed. Example 20.8 In the market of Figure 20.4, find the possible no-arbitrage prices for a call option with exercise date 1 and strike price 110. Solution. Add the option as another assetS 2 with a price of𝜋 at time 0. For a martingale measure which has probabilityp of an upward movement and probabilityq of staying the same,wehave120p +100q+80(1−p−q) = 100implyingthat2p +q= 1,sothat p< 1∕2. Applyingthemartingaleconditionforthenewassetgives 𝜋 = 10p,whichmeansthatwemust have 0<𝜋<5. Conversely, all such values are admissible since for any such𝜋 we obtain a martingale measure for the enlarged market by taking p= 𝜋∕10, q= 1−𝜋∕5. In some cases, specifying the price of certain contingent claims will determine others. To illustrate, having added the additional asset and having specified𝜋 in Example 20.8, the resulting martingale measure is unique, so we have in effect completed the market. The price specified for the option will determine unique prices for all other contingent claims. 20.12 The Black–Scholes–Merton formula The discrete-time model illustrates many of the fundamental principles of pricing contingent claims. There is a limitation, however. To be at all close to a realistic model, we would need an enormously large number of time periods and the computations will quickly become intractable.Thisisapparentjustfromlookingatthecasesfor N = 2oftheprevioussection.For actualcomputationandgreaterrealism,thepreferredmethodistousecontinuous-timemodels, which involve more advanced mathematical machinery. The relatively simple calculations effected by solving linear equations above will be replaced by solving differential equations. Sums of random variables are replaced by integrals of random variables, a concept that is technicallyverycomplex.Thecharacterizationsofarbitrage-freenessandcompletenesswhich we gave can be generalized to continuous-time settings but the proofs are far more difficult. A rigorous development of this is beyond the scope of the book, However we do want to 362 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS investigate theBlack–Scholes–Mertonformula. This is a pivotal result which in fact initiated much of the modern research into stochastic models in finance. In this section, it is convenient to use the risk-free force of interest𝛿 =l o g(1+r) instead ofr. We consider again the case of a financial market consisting of the bank account, and a single stock whose price at timetis S(t). The difference is that we now allow trading at any time,andmoreover,weallowassetpricestovarycontinuously.Wethereforemustmodel S(t) asacontinuous-timestochasticprocessandwillinfactchooseageometricBrownianmotion process, as given in Section 18.7. That is, for some constants𝜇 and𝜎, S(t) = S(0)e 𝜇t+𝜎B(t), whereB(t) is a standard Brownian motion. The problem is to find the price of a European call option with exercise dateN and strike price K. One approach is to make use of the results that we already know for the discrete- time model. Suppose we can find a probability measureQunder whicĥS(t) is a martingale. Then we can approximate our process with our discrete time multiple period binomial model with periods of length 1∕m, u= e 𝜎 √ 1∕m,d= e−𝜎 √ 1∕m and with risk-free force of interest= 𝛿∕m, provided we takemsufficiently large. This should seem plausible since the log of our binomial process is a random walk. In this discrete setting, the price of the option is given in Equation (20.9). Putting in all the parameters and taking limits asmgoes to∞,w ea r r i v ea t the celebrated Black–Scholes–Merton formula: Option price= S(0)Φ(𝛼 +𝜎 √ N)−Ke−𝛿NΦ(𝛼), (20.19) where 𝛼 = log(S(0)∕K)+(𝛿 −𝜎2∕2)N 𝜎 √ N , (20.20) and Φ is the c.d.f. of the standard normal distribution. An alternative method to derive the formulaistodirectlycalculate e−𝛿NEQ[(S(N)−K)+].Thisismorestraightforward,involving basiccalculus, although thecalculation issomewhat involved. For both ofthesemethods, we will omit the detailed derivations. Notethattheparameterstodeterminetheoptionpricearetherisk-freeforceofinterest 𝛿, the strike priceK, the durationN, and the quantity𝜎. The latter is a measure of the tendency for prices of the underlying stock to vary and is known as thevolatility of the stock. Note however that the formula does not depend on𝜇. We comment more on this below. The resulting formula seems somewhat formidable, but we can provide some motivation. Thefollowingisnotintendedasarigorousexposition,butmoretoprovideamethodofremem- bering and understanding the structure of the formula. First consider the random variable X =l o g[(S(N)∕S(0)]. What is its distribution under Q? From the definition of Brownian motion, we know that under our original probability measure, it is normal with mean𝜇N and variance𝜎 2N.I fw e make the assumption that it is still normal underQwith the same variance, and calculate the resulting meanM, we know from Equation (A.58) that EQ[S(N)∕S(0)] = eM+(𝜎2∕2)N. THE BLACK–SCHOLES–MERTON FORMULA 363 But, sincêSN is a martingale with respect toQ, S(0) = E(̂SN) = e−𝛿NE(SN), so that EQ[S(N)∕S(0)] = e𝛿N, leading to M = (𝛿 −𝜎2∕2)N. (20.21) Weconcludethatunder Q,wehaveanewdriftparameter(𝛿 −𝜎2∕2)whichiscompletely independent of the original drift parameter𝜇. This is analogous to the observation in the simpleone-periodbinomialmodelwheretheoriginalprobabilityplayednoroleintheoption price and it was only the risk-neutral probability that mattered. Now let us ask, what is the probability underQthat the buyer of this option will exercise it? This will occur ifS(N) > K. The probability of this is the same as the probability that X ≥ log(K∕S(0) which equals the probability that−X ≤ log(S(0)∕K), which by the results calculated above is Φ ( log(S(0)∕K)+(𝛿 −𝜎 2∕2)N 𝜎 √ N ) . (20.22) Given our above assumption regarding the distribution ofX, we have now identified the term Φ(𝛼) in the Black-Scholes-Merton formula as the probability under the risk-neutral measure that the option will be exercised. Now consider the following rather naive reasoning to arrive at the option price. If we exercise the option, we will have an expected gain at timeN of E[S(N)−K)] and this will have a present value ofE[S(N)−K]e−𝛿N. Multiply this by the probability of exercise to get S(0)Φ(𝛼)−Ke−𝛿(N)Φ(𝛼). This looks something like the actual Black–Scholes–Merton formula, but unless𝜎 = 0, the coefficient ofS(0) differs. Our naive approach, however, ignores the nature of the option contract. With positive values of𝜎, the stock price at timeNwill vary, and could be above or belowtheexerciseprice.Calculating E(S(N)−K)willincludenegativevalueswhentheprice isbelow K,butthereisnolossinthesecasessincetheoptionwillnotbeexercised.Thenaive formula therefore understates the true price. Another way of expressing this is to note that what we really want is the expected value ofS(N)−K, given that the option is exercised. To correct this understatement, it turns out that the coefficient of S(0) must be increased fromΦ(𝛼)t oΦ(𝛼 +𝜎 √ N). This makes sense as we would expect that this correction should increase as the variability in the stock price and length of period increase. For further insight and verification, we can directly verify the formula forN = 0. In this case, Φ(𝛼) =Φ (−∞) = 0i fS(0) < K, and the formula gives an option price of 0, which it should,while Φ(𝛼) =Φ (∞) = 1ifS (0) > Kandtheformulagivesanoptionpriceof S(0)−K which it should. There is still another important feature involving the two terms in the Black–Scholes – Mertonformula,whichwenowpresent.Startwiththefollowingquestion.Forthecalloption as above, what is its value at any timet,0 ≤ t≤ N? By the same argument as above, the 364 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS value will be given by the same formulas as above except that in both (20.19) and (20.20), S(0) is replaced by byS(t) which is the stock price at timetand N is replaced byN−t,t h e remaining duration. It turns out that the two terms of this formula yield a replicating strategy for the option, and therefore a hedging strategy, in terms of holdings of the stock and the bank account. To explain this, we will take new units for the bank account, by adopting a slightlydifferentbutequivalentpointofview,Wecanviewthebankaccountasaninvestment in risk-free zero-coupon bonds maturing at timeN. A unit of this asset will be a bond with face amount K, so that its value at time 0= K(1+r) −N.L e t𝛼t denote the value of (20.20) withS(0)replacedby S(t)and Nreplacedby N−t.Thereplicatingtradingstrategyistohold Φ(𝛼t +𝜎(N−t)) units of the stock, and carry a short position ofΦ(𝛼t) units of the bonds we just described, at any timet. At each timet, the value of this portfolio will be precisely the value of the option as given above and it will reach the right amount at timeN. In the case where the option is to be exercised the final portfolio will consist of 1 unit of stock, and a short position of 1 bond. In the other case, the portfolio will be reduced to zero units for both assets. It can be observed that the idea behind this strategy is similar to what we observed in thecalloptionofExample20.2,wherethereplicationisaccomplishedbybuyingstockasthe price increases, or selling as the price declines. One must also show that this strategy is self-financing, using an appropriate modification of this definition to apply to continuous time. The idea is that at each instant, the amounts received from selling one of the assets is exactly what is required to buy the other. A precise definition is based on the formulation given in Exercise 20.3, with the differences replaced by differentials. This is not easy, however. The problem is that these are differentials of stochastic processes rather than deterministic functions, and a rigorous presentation involves some knowledge of the subject known as stochastic calculus. Remark In real life of course this replication by continuous rebalancing is not possible. One could try to get close by very frequent rebalancing but there is no guarantee that this discrete-timeapproximationisself-financing.Itcouldwellrequireadditionalamountsofcash or release such. But one can hope that if our model is sufficiently accurate that these extra amounts are relatively small, sothat theoption could bereplicated forsomething closetothe Black–Scholes–Merton price. To conclude this section, we note that while the Black–Scholes–Merton formula has persistedasthemaintoolforoptionpricing,itisbasedonseveralsimplifyingassumptions.One such assumption is that of the log-normal evolution of stock prices. There has been evidence to show that this is not completely realistic, and alternative models have been investigated. Another assumption is that both the risk-free force of interest and the volatility are constant and known, as these must be inserted into the formula to obtain numerical results. More realistic models have been proposed where these quantities are both considered as random. 20.13 Bond markets 20.13.1 Introduction We now return to the discrete-time case. In this section, we deal with markets where the assetsarerisk-freezero-couponbonds.ThesewerealreadyintroducedinSection2.12,where BOND MARKETS 365 we considered forward prices, but here we want to consider the actual prices which will be random variables. A major result will be to give an appropriate version of 2.1 to ensure no arbitrage in the case of stochastic discount functions. Our market will consist ofN+1 assets where forn≠ 0, assetn is a zero-coupon bond maturing for 1 at timen. We will defineS 0 later. We will denote the random variableSk(n)b yv(k,n). Fork≤ n, v(k,n) is the price you would pay at timek to receive 1 at timen, which ties in well with our original use of this notation in Chapter 2, as well as our notation for forward prices. These are not the forward prices however, but the actual prices. Of coursev(0,n) is a real number and the same as the forward pricẽv(0,n), since at time 0 we know the prices of the bonds. But future prices are unknown and therefore modelled as random variables. Aquestion which may come tomind now iswhether the fundamental Equation 2.1could holdforthesequantitiesonlyinterpretedasamultiplicationofrandomvariables.Theanswer is no. Observe, for example, that with randomness v(0,m) ≠ v(0,n)v(n,m), since the left side is a real number while the right side is truly random. Now in order to avoid a completely trivial situation, we cannot assume a constant deter- ministicrisk-freerate raswedidbefore.Beforediscussinghowwemodifythisidea,wewant to recall how risk arises from investment in risk-free assets like bonds, (elaborating on the discussion at the end of Section 2.10.3). Note first that unlike a stock, where a value on any futuredateisunknown,thepayoffonarisk-freebondisabsolutelycertainifheldtomaturity. Theriskarisesifonewantstobuyorsellbeforethatdate.Forexample,iftherisk-freeinterest todayis5%,thenthepriceofa3-yearbondwillbe(1 .05) −3 = 0.864.Ifattime1,therisk-free interest rate has risen to 0.08, then the value of the bond drops to (1.08)−2 = 0.857. Buying long-termbondsthereforecarriestheriskofariseininterestrates,whichcanlowertheprice. However, in our present model, where trading occurs only at integer times, buying bonds whichmaturein1perioddoesnotcarryanyrisk.Thisleadsustodefinethebankaccountby S 0(0) = 1, S0(n) = [v(0,1)v(1,2) ⋯ v(n−1,n)]−1,n≥ 1. (20.23) Ourbankaccountisformedbystartingwith1attime0,usingthattobuyabondmaturing at time 1, then taking the proceeds ofv(0,1)−1 to buy a bond maturing forv(0,1)−1v(1,2)−1 at time 2, and to continue rolling over the account into a new 1-year bond each year. In the case of a constant and deterministic interest rate ofr, we would havev(k,k+1) = (1+r) −1, and this definition ofS0 reduces to the one given before. Note that under this new definition,S0(k) is not a definite amount, but strictly random for k> 1. Using the bank account, we can extend the definition ofv(k,n) to all ordered pairs (k,n) by defining v(k,n) = S0(k)∕S0(n), k> n, since this is the amount accumulated at timek, by taking 1 from a bond maturing at timen and placing it in the bank account. Note that with this definition, all prices at the final time N are determined by the previous values ofS 0(n) (and of course the fact thatV(N,N) = 1). 366 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS This means that we can model our bond market with a treegoing up to only timeN−1. (See Example 20.9 below.) 20.13.2 Extending the notion of conditional expectation For our analysis, we will need some additional results aboutk-determined random variables, We will in fact investigate this idea in more generality and further extend the discussion in Section A.8, since the same ideas are needed in Chapter 24. Suppose we are given a sample spaceΩ,aprobabilitymeasure PonΩandapartition Π={B 1,B2,…Bn}ofΩ intopairwise disjointsetswithunionequaltoallof Ω,suchthatforall i,P(Bi) > 0.Foranyrandomvariable W on Ω, define a random variableEΠ(W) as follows. For𝜔 ∈ Bi, EΠ(W)(𝜔) = E(W|Bi). Toillustrate,inthemodelofSection20.8,fix kandtakethepartition Πwhichconsistsof all sets𝜈◦ where 𝜈 is an admissiblek sequence. (For example, in the market of Figure 20.2 if we takek= 1, the partition will consist of the three sets {0,1}◦,{0,2} ◦,{0,3} ◦.) Then EΠ(W)i sj u s tEk(W) as defined in that section. We summarize the facts that we need in the following theorem. Theorem 20.6 TakeanyrandomvariablesV,W andascalarc. (a)EΠ islinear.Thatis EΠ(V+W) = EΠ(V)+EΠ(W), EΠ(cV) = cEΠ(V). (b)SupposethatW isarandomvariablewhichisconstantoneachsubsetof Π.Then EΠ(WV) = WEΠ(V). (c)Supposethat Π′ isa finer partitionthan Π whichmeansthateverysetin Π isaunion ofsetsin Π′.Then EΠ[E′ Π(W)] = EΠ′[EΠ(W)] = EΠ(W), Proof. (a)Thisfollowsdirectlybyapplying(A.23)and(A.8)toeachsubset Bi ofthepartition equipped with the probability measureP(⋅|Bi) (b) As in (a) apply (A.8) to each set of the partition. (c) It is clear from the definitions that if a random variable is constant on each subset of Π,thenapplying E Π toitwillleaveitunchanged.Now EΠ(W)isconstantoneachsubsetof Π therefore constant on each subset of the finer partitionΠ′. It follows thatEΠ′[EΠW] = EΠW. To show the other order of composition, choose any setAof the partitionΠ and letAbe the union of setsB1,B2,…Bm where eachBi is a set of the partitionΠ′. To simplify the notation denote EΠ(W)b yZ. Now by definition,Ztakes the constant value ofE(W|Bi) on each setBi so that clearly E(Z|Bi) = E(W|Bi). BOND MARKETS 367 Now apply (A.29) to the sample spaceAequipped with the probability measureP(⋅|A). We have that E(Z|A) = m∑ i=1 E(Z|Bi)P(Bi) = m∑ i=i E(W|Bi)P(Bi) = E(W|A) which shows that EΠ(Z) = EΠ(W) It is of interest to look at the extreme cases. If we takeΠ to the finest possible partition wherethesetsaresingletons,then EΠ(W)isjust W.Ifwetake Πtobethepartitionconsisting of just one set, namely the whole space, which is the least fine partition, thenEΠ(W)i sj u s t the usual expectationE(W). The latter observation leads to some results of interest which we use in Chapter 24. Immediately from part (c) of the Theorem 20.6, for anyW, E[EΠW] = E(W) (20.24) More generally, ifZ is any random variable which is constant on the sets of the partitionΠ, then from part (b) of Theorem 20.6 E(ZW) = E[EΠ(ZW)] = E[ZEΠW]. (20.25) 20.13.3 The arbitrage-free condition in the bond market Return to our bond market with the assumption that for allk, the random variablev(k,n)i s k-determined. What is the condition onv(k,n) to ensure that the market is arbitrage-free? Lets go back to the deterministic setting of Chapter 2 for a moment. Note that if a special case of Equation (2.1) holds, namely v(k,n) = v(k,k+1)v(k+1,n) (20.26) forallnonnegativeintegers k≤ n,thenastraightforwardinductionargumentshowsthat(2.1) willholdforallnonnegativeintegers k≤ m≤ n.(Example20.9belowwillmakeitclearwhy we begin with Equation (20.26) in place of the more general Equation (2.1)) We saw above that (2.1) does not hold when considered as a statement about random variables, but could it possibly be that this revised version will be valid? The answer is still no, since the left hand side isk-determined, but in general, sincev(k+1,m) is onlyk+1- determined, the right side will only bek+1-determined. It is plausible, however, that the following natural modification of our statement holds. Namely v(k,n) = v(k,k+1)E kv(k+1,n). (20.27) It turns out in fact that Equation (20.27) is the correct condition to prevent arbitrage. Theorem 20.7 If we can find a probability measure Q onΩ such that Equation (20.27) holdsforallnonnegativeintegersk < n≤ N,thenthegivenbondmarketisarbitrage-free. 368 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS Proof. We will show that underQ, eacĥSn satisfies (20.11) and we can then apply Theo- rem 20.4. We know from the definition ofS0 that S0(k+1) = S0(k)v(k,k+1)−1. Moreover, ̂Sn(k+1) = v(k+1,n) S0(k+1) = v(k+1,n)v(k,k+1) S0(k) . Now S0(k) is certainly a k-determined random variable, being a product of k-determined randomvariables,andsothereforeis S0(k)−1.UsingTheorem20.6(b)andinvokingEquation (20.27) Ek(̂Sn(k+1)) = Ek[v(k+1,n)]v(k,k+1) S0(k) = v(k,n) S0(k) = ̂Sn(k) completing the proof. 20.13.4 Short-rate modelling We now deal with a problem that is different from that in previous sections. Instead of being giventheassetprices,andaskedifthemarketisarbitrage-feeornot,wearegivensomeprices and want to determine other prices to satisfy the arbitrage-free condition. For example, refer back to Equation (2.6). This shows in effect that in an deterministic setting, the prices for bonds of one period determine those for all periods. We do in fact have a stochastic version of this formula obtained by taking expectations. Suppose we have a probability measure Q on Ω and random variables v(0,1), v(1,2), …v(N−1,N), where v(k,k+1) is k-determined. We extend this to all our prices by the rule that fork< n, v(k,n) = E k[v(k,k+1)v(k+1,k+2)v(k+2,k+3)…v(n−1,n)]. (20.28) This will indeed result in an arbitrage-free market, since v(k,k+1)Ek[v(k+1,n)] = v(k,k+1)Ek[Ek+1[v(k+1,k+2)…v(n−1,n)]] = v(k,k+1)Ek[v(k+1,k+2)…v(n−1,n)] = Ek[v(k,k+1)v(k+1,k+2)…v(n−1,n)] = v(k,n). (20.29) establishingcondition(20.27).Hereweusedthedefinition(20.28)inthefirstandlastequality, ForthesecondinequalityweusedTheorem20.6(c)notingthatforany k-admissiblesequence 𝜈,t h es e t𝜈 ◦ is a disjoint union of sets𝜈◦ i where each𝜈i is ak+1-admissible sequence – simply add on all possible choices for the last element. So askincreases, the partitioning by k-admissible sequences gets finer. Finally, we use Theorem 20.6(b) for the third inequality. BOND MARKETS 369 Example 20.9 Take N = 3. As remarked above, we need only consider a two-period model. We will take the binomial model, and letQ be the probability measure that assigns equal probability of 1∕4 to each of the four elements inΩ={UU ,UD,DU,DD}. Define the random variablesv(0,1), v(1,2) and v(2,3) by v(0,1) = 0.7, v(1,2)(U) = 0.8 v(1,2)(D) = 0.6.v(2,3)(UU) = 0.9,v(2,3)(UD) = 0.7,v(2,3)(DU) = 0.7,v(2,3)(DD) = 0.5. (a) Use (20.28) to find the distribution underQof the other random variablesv(k,n)f o r k< n, which will make the market arbitrage-free. (b) Show that it is not necessarily true thatv(k,n) = v(k,m)Ekv(m,n)i fm≠ k+1. Solution. (a) The other random variables arev(0,2), v(0,3), v(1,3)which we will calculate directly from Equation (20.28) v(0,2) = E[v(0,1)v(1,2)] = 1 2(0.7×0.8+0.7×0.6) = 0.49. The 2-determined random variablev(0,1)v(1,2)v(2,3) takes the value of 0.7×0.8×0.9 on UU, the value of 0.7×0.6×0.7 on DU, the value of 0.7×0.8×0.7 on UD, and the value of 0.7×0.6×0.5 on DD. Each of these paths has probability 1∕4, so v(0,3) = E[v(0,1)v(1,2) v(2,3)] = 1 4[0.504+0.294+0.392+0.210] = 0.35 Nowv(1,2)v(2,3)takesthevalueof0 .8×0.9onUU andthevalueof0.8 ×0.7onUD so v(1,3)(U) = 1 2(0.72+0.56) = 0.64 Also, v(1,2)v(2,3) takes the value of 0.6×0.7o nDUand the value of 0.6×0.5o nDDso v(1,3)(D) = 1 2(0.42+0.30) = 0.36 (b) v(0,2)E2[v(2,3)] = (0.49)(0.7) = 0.343 ≠ v(0,3) Totiethisexampleinwithpreviousmaterial,thereadermayfinditinstructivetoreproduce the figures shown in Figures 20.7 and 20.8. Figure 20.7 shows all the asset prices for all four assets,Figure 20.8 shows thevalues of̂S k(n)f o rk= 1,2,3. The values of̂S0(n)are of course all equal to 1. Instead of specifying the one-period bond prices, we could equivalently have specified the interest rates as we did in Chapter 2, except that nowik = v(k,k+1)−1 −1, which is a k-determined random variable. We have shown that we can model stochastic interest rates in much the same way as we did in the deterministic model, that is, by choosing the one period rates first. For this reason, this procedure is sometimes referred to as short-rate modelling. What may be surprising at first, is that in the stochastic case the result in far from unique. We can indeed specify any probability measure we want for eachi k, and achieve an arbitrage-free bond market. The procedure doesnottellushowtochoosetherandom variables ik andtheprobabilitymeasure 370 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS 1, 0.7, 0.49, 0.35 0.7‐1, 1, 0.8, 0.64 0.56‐1, 0.8‐1, 1, 0.9 0.56‐1, 0.8‐1, 1, 0.7 0.42‐1, 0.6‐1, 1, 0.7 0.42‐1, 0.6‐1, 1, 0.5 0.7‐1, 1, 0.6, 0.36 Figure 20.7 Example 20.9. Values ofSk(n), k= 0,1,2,3 Q. In practice, this can be motivated by trying to satisfy other conditions that one wants to impose.Forexample,attime0,oneknowsthevalues v(0,n).ButaswesawinExample22.7, thevalues v(0,n)aredeterminedfromtheone-periodrates.Aproblemofsomeinterest,which we will not deal with here, is to choose the distributions of thei k so as to recover specified values ofv(0,n). 20.13.5 Forward prices and rates Wenowexpandonourdefinitionofforwardbondprices,whichweintroducedinSection2.12 Suppose we are given a financial bond market as above. Forj≤ k≤ n,w el e t 0.7, 0.049, 0.35 0.7, 0.56, 0.448 0.7, 0.56, 0.464 0.7, 0.56, 0.372 0.7, 0.42, 0.294 0.7, 0.42, 0.210 0.7, 0.42, 0.252 Figure 20.8 Example 20.9. Values of̂Sk(n), k= 1,2,3 BOND MARKETS 371 ̃vj(k,n) = the forward price to be paid at timej for a 1-unit zero-coupon bond issued at timekand maturing at timen. This will be aj-determined random variable, since it is random depending on the state of nature up to timej. The quantitỹv(k,n) defined in Section 2.12 could be labeled as̃v0(k,n). ThesameargumentasusedinSection2.12showsthattopreventarbitrage,wemusthave ̃vj(k,n) = v(j,n) v(j,k). Indeed,ifthereisanysamplepointatwhichthereisadiscrepancybetweenthesetworandom variables,anindividual couldwaitandseeifthisparticularsamplepointmaterializedattime j, which would be known by the fact that the random variables arej-determined. One could then follow one of two strategies, depending on which way the inequality went, to achieve a sure profit at timen. In particular̃v j(j,n) = v(j,n) which is clear from the definition. In place of the forward prices, we can describe the information available from the bond prices at timejby interest rates as we did before. Forj≤ k, we define aj-determined random variableij(k), theforwardinterestrate for contracts at time timejapplicable to time periodk tok+1, by ̃ij(k) = (̃vj(k,k+1))−1 −1 Note thatik(k) = ik. In the deterministic caseij(k) = ik for allk. It follows immediately from the definitions that v(k,n) = ̃vk(k,k+1)̃vk(k+1,k+2)…̃vk(n−1,n) = [(1+ ̃ik,k)(1+ ̃ik,k+1)…(1+ik,n−1)]−1 (20.30) so that bond prices are determined by the forward interest rates. This suggests an alternative procedure to short-rate modelling for bond prices. The method is to model all the forward interestratesinplaceoftheshortrate i k.Inthiscase,theforwardratesattime0areallknown, sothemodelwillclearlyreproducetheobservedpricesattime0.Onecannot,however,choose thedistributionsoftheforwardratesarbitrarilyasfortheshortrates.Certainconditionsmust be imposed to ensure that the model is arbitrage-free. (See Exercise 20.12). This is another topic that is beyond our scope for further elaboration. 20.13.6 Observations on the continuous time bond market We have concentrated on the discrete time setting for bond market, but many applications involve a continuous time framework. In this section, we give a very quick overview. It is intended mainly to bridge the gap and prepare those who wish to look at other sources and compare the material there with what we have done above. Our market now will consist of zero-coupon bonds maturing at any timet, with a price at times,ofv(s, t).Thiswillbea s-determinedrandomvariableunderanappropriateextensionof thisdefinitiontocovercontinuoustime.Aswellwecandefineanextensionoftheconditional expectationE s. We will not go into details. 372 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS In place of the forward and spot rates of interest, one often wants to deal with the corresponding forces of interest, generalizing the quantity𝛿(t) to a stochastic setting. The forward force of interest, for contracts at timesapplicable to timetis given by ̃𝛿s(t) =− 𝜕 𝜕t logv(s,t), an s-determined random variable. In the deterministic case, when we can assume that our basic Equation (2.1) holds, then log(v(s,t) =l o gv(t)−log v(s) and all forward forces will be equal to𝛿(t) as defined in (8.3). Knowing the forward forces determines bond prices uniquely, since by the definition v(s,t) = e−∫ t s ̃𝛿s(r)dr, which we can view as a stochastic version of Equation (8.4) and as a continuous version of Equation (20.30). We can also define a continuous and stochastic version of the short rate. Let𝛿(t) denote ̃𝛿 t(t). This perhaps requires some care, since the partial derivative applies only to the second variable. Precisely 𝛿(t) = −lim h→0 logv(t,t+h) h , which will be at determined random variable, and one which agrees with our Chapter 8 definition in the deterministic case. As in the discrete time case, this will not uniquely determine bond prices in a stochastic setting. Given any stochastic process for𝛿(t), we can produce an arbitrage-free bond market by taking v(s,t) = E s[e−∫ t s 𝛿(r)dr], under a suitable interpretation of an integral of a random integrand. Notes and references This chapter constitutes a basic introduction to financial markets. For more comprehensive coverage of the basic concepts, see Hull (2014) or McDonald (2012). Readers particularly interestedinthemathematicsofcontinuous-timemodelscanconsultBj ¨ork(2009)orEtheridge (2002) for sources that are not overly advanced. Formanyyears,formula(20.19)wasreferredtoastheBlack–Scholesformula,recognizing the authors of the original paper that was published. Recently, many writers have added the name of Robert Merton, who made important contributions to developing and extending the ideas behind this result. A proof of the extension result on hyperspaces can be found in Steland (2012), Theo- rem 2.4.5. Thosereadingthefinancialliteratureshouldbeawarethattheterminologycandifferfrom the actuarial conventions that we have adopted. Financial economists often use the wordrate of interest to mean the continuously compounded rate, which we have called the force of interest. EXERCISES 373 Exercises 20.1 A one-period financial market has in addition to the bank account, one risky asset. The price at time 0 is 50, and at time 1, the price will either be 52 or 55 each with positiveprobability.Forwhatvalueoftherisk-freeinterestrate rwillthismarketbe arbitrage-free? 20.2 For the option contract introduced in Section 20.5, find an arbitrage opportunity if the option price is 11. 20.3 For the option contract introduced in Section 20.5, find the price ifr is changed to (a) 11%, (b) 9 %. Can you explain why the prices change in this way? 20.4 Show that a trading strategy is self-financing if and only if, for 0≤ n< N, V(n+1)−V(n) = M∑ j=0 𝛼j(n)(Sj(n+1)−Sj(n). 20.5 Assume the data of Example 20.2. Find the self-financing trading strategy for a 2-year American put option with a strike price of 106. Compare the price with the corresponding European put option. 20.6 Consider the financial market with N = 1, M = 1, r= 0, S 1(0) = 100, S1(1) = 130,110, or 80, all with positive probability. (a) Find all possible risk-neutral probability measures. (b) Consider a 1-year call option with a strike price of 100. (This is known as an ‘at-the-money’option).Findtherangeofpossiblepricesforthisoptionthatwill avoid arbitrage. (c) If the price of the option in part (b) is 12, find an arbitrage opportunity. 20.7 Consider the two risky asset market of Figure 20.6 withr= 0; (a) UseTheorems20.4and20.5todecidewhetherornotthismarketis(i)arbitrage- free, (ii) complete. (b) Describe the subspacesLandL 0. 20.8 In Figure 20.3, find the slopes of the two lines representingL0 in terms ofu, d and r. 20.9 WhenpricingoptionswiththeBlack–Scholes–Mertonformula,describehowoption prices change, as changes occurs in volatility, strike price, duration, and risk-free force of interest. Use the formula and put-call parity to compute put and call prices for 3-month European options, on a stock selling now for 100 with a strike price of 110,100and90respectively,assuming 𝜎 is0.2andtheforceofinterestfora1-year periodis0.06.Verifyyourconclusionsabovebyrepeatingthecalculationsforother values of𝜎, 𝛿 and duration. 374 INTRODUCTION TO THE MATHEMATICS OF FINANCIAL MARKETS 20.10 Foracertainstock,thepriceofacalloptionis7.80andthepriceofthecorresponding put option with the same strike price and expiration date is 1.50. In each of the following scenarios, find the new price of the put option. (a) A change in volatility raises the call price to 11.30. (b) The stock price at time 0 is 100. A 10% increase in the strike price lowers the call price to 5.00. (c) Therisk-freeforceofinterestisdoubledandthetimetoexpirationiscutinhalf, which combine to lower the call price to 7.00. 20.11 Consider a 6-month European call option on a stock now selling for 100, with a strike price of 97. You are given that𝜎 = 0.25 and the force of interest for a 1-year periodis0.10.Youwouldliketoreplicatethisoptionbyatradingstrategyinvolving holdings in the stock and risk-free zero-coupon bonds of face amount 97, maturing in 6 months. You plan to use the Black–Scholes–Merton formula. (a) What is the initial portfolio at time 0? (b) At the end of 2 months, the stock price has risen to 105. What is your portfolio now? (c) At the end of 4 months, the stock price has fallen to 95. What is your portfolio now? 20.12 Give a direct verification for the Black–Scholes–Merton formula in the case that 𝜎 = 0. 20.13 Verify directly from Figure 20.8 that the market of Example 20.9 is arbitrage-free and complete. 20.14 Redo Example 20.9 only assuming that the probability of an upward move is 3∕4 and that of a downward move is 1∕4. 20.15 A bond market has two transitions,U and Dfrom time 0 to time 1. You are given the following forward prices: (i) ̃v(0,1) = ̃v(1,2) = ̃v(2,3) = 0.7; (ii) ̃v 1(1,2) and̃v1(2,3) both take the value 0.8 onUand 0.6 onD. (a) IfQ(U) = Q(D) = 0.5, show that Equation (20.27) is not satisfied. (b) Show that there is no probability measureQ for which Equation (20.27) is satisfied. 20.16 ConsiderthebondmarketwithpricesasgiveninFigure20.7.Consideracalloption onabondmaturingfor1000attime3,withexpirationdatetime2.Findthepriceof the option if the strike price is (a) 900, (b) 650. *20.17 Auniversallifecontractprovidesthatyouraccountwillbecreditedwithaminimum of 4% interest, up to a maximum of 10% interest. Describe the options present in this arrangement. Part IV RISK THEORY