Instruction:
What is the probability that
Z = 5?

Response: Appendix A review of probability theory Thisappendixprovidesareviewofthebasicprobabilitytheoryusedinthisbook.Inaddition, more specialized topics in probability theory, will appear in the relevant chapters as they are needed. It is expected that most readers will be at least somewhat familiar with this material, so the pace is fairly rapid, with few examples or derivations. A.1 Sample spaces and probability measures We model the results of a random experiment by a setΩ, known as asample space, where each point ofΩ corresponds to a possible outcome. For example, if we throw a pair of dice, thesamplespacecouldconsistofthe36orderedpairs( a,b)where aandbtakevaluesfrom1 to6.Acombinationofoutcomes,knownasan event,isrepresentedbyasubsetof Ω.Suchan event occurs if any of the outcomes in the subset occur. In the above example, the event that thetotalonthediceis10,wouldberepresentedbytheset{(4,6),(5,5),(6,4)}.Fromagiven collection of events, familiar set operations can be applied to build other events. The union of events, denoted with the symbol∪, gives us the event that occurs if any one of the given events occur. The intersection of events, denoted with the symbol∩, gives us the event that occurs if all of the given events occurs. The complement of an eventA, denoted byA c,g i v e s us the event that occurs ifAdoes notoccur. Two events are said to bemutually exclusiveif the occurrence of one means that the other cannot occur. These are represented by subsetsA andBthat aredisjoint, that is,A∩B=∅ (the empty set). To each eventAwe assign a numberP(A) in the interval [0,1], known as theprobability ofA.Thismeasureshowlikelyitisfortheevent Atooccurinasingletrialoftheexperiment. Theassignment P(A)mustsatisfythefollowingfundamentalrule:Givenafiniteorcountably FundamentalsofActuarialMathematics , Third Edition. S. David Promislow. © 2015 John Wiley & Sons, Ltd. Published 2015 by John Wiley & Sons, Ltd. Companion website: http://www.wiley.com/go/actuarial 494 A REVIEW OF PROBABILITY THEORY infinite sequence A1,A2,…,An,… of pairwise disjoint events – that is,Ai ∩Aj =∅ for i≠ j– then P [ ⋃ i Ai ] = ∑ i P(Ai). (A.1) Moreover, we require thatP(Ω) = 1. These requirements can be motivated by adopting the relative frequency interpretation of P, wherebyP(A) gives the expected proportion of times that the eventA will occur if the experiment is repeated a sufficiently large number of times. We often refer toPas aprobabilitymeasure on Ω. The following are some simple, frequently used consequences of (A.1), P(∅) = 0, A⊆ BimpliesP(A) ≤ P(B), P(Ac) = 1−P(A), (A.2) and P(A∪B) = P(A)+P(B)−P(A∩B). (A.3) We now discuss a technical difficulty that arises when the sample space is not finite or countably infinite. In this case, postulating thatP(A) should be defined for all subsetsA is overly restrictive, and would prevent one from finding suitable probability measures in many cases.Accordinglyweonlyrequirethat P(A)bedefinedwhen Abelongstoacertainspecified collectionofsubsets,whichwedenoteby .Wedo,however,requirecertainrestrictionson  toensurethatcountablefamiliesofeventscanbecombinedinthefamiliarwayswedescribed above. These requirements are as follows: (a) Given any finite or countably infinite sequenceA 1,A2,…,An,… of sets in, (i) their union is in, (ii) their intersection is in. (b) For anyA∈ , the complementAc ∈ . (c) The whole setΩ∈ . A collection of subsets with these properties is known as a𝜎-field. (Since (a)(ii) follows from (a)(i) and (b), while (a)(i) follows from (a)(ii) and (b), many authors omit one of (a)(i) or (a)(ii) in the definition of a𝜎-field.) As a consequence of (a)(i) we know that the left hand side of (A.1) makes sense. To summarize, in modelling a random experiment, we choose a sample spaceΩ, a prob- ability measurePand a𝜎-field . WhenΩ is finite or countably infinite, the collection is invariably taken to be all subsets. In the remaining sections of this appendix, we assume a fixedΩ,Pand. RANDOM VARIABLES 495 A.2 Conditioning and independence Ifwearetoldthatacertainevent Bhasoccurred,weneedtoreassessprobabilities.Assuming P(B) > 0, this is done by defining a new probability measure, denoted byP(⋅|B), as follows: P(A|B) = P(A∩B) P(B) , A∈ . Weoftenrefertothelefthandsideas‘theprobabilityof AgivenB’.Thisnewmeasureassigns probability0toeventsdisjointfrom B,andfor A⊆ Bitassignstheprobability P(A)∕P(B).In a sense, we can view this as changing the sample space fromΩ to B, and then dividing by a constant to ensure that the probability of the entire space is 1. The symbolP(A|B1,B2,…,Bn) denotes the probability ofAgiven that all of the events Bi have occurred. That is, it equalsP(A|B) whereB=∩ n i=1 Bi. We say thatAandBare independentif P(A|B) = P(A). In other words, knowing thatB occurred does not affect our original assessment of the likelihood ofA. It is convenient to write this condition in the symmetric form P(A∩B) = P(A)P(B), whichshowsimmediatelythatwealsohave P(B|A) = P(B).Moreover,thisformmakessense if either set has probability 0, in which case we automatically have independence. More generally, we define a collection of events (possibly infinite) to be independent if, given any finite subcollectionA1,A2,…,An, P [ n⋂ 1=i Ai ] = P(A1)P(A2)⋯ P(An). (A.4) A.3 Random variables Randomvariables are,intuitively,numericalquantitiesassociatedwitharandomexperiment. For example, we toss 100 coins and count the number of heads. Formally, we represent the random variable by a real-valued functionX defined onΩ, where for𝜔 ∈Ω , X(𝜔)i s the number obtained when the outcome of the experiment is𝜔. The technical restriction we referred to above whenΩ is not finite or countably infinite also puts restrictions on the possible functions we consider. Given a random variable, traditionally denoted by a capital letterlike X,andasubset Aofrealnumbers,wewouldliketofindtheprobabilitythat Xtakes a value in the setA. This is not possible in our model if the event {𝜔∈Ω : X(𝜔) ∈ A} is not in the collection of events that we are able to compute probabilities for. Accordingly, in the definition of a random variable we postulate that at least forA of the form (−∞,r], the corresponding event is in, so we can always compute the probability thatX is less than or equal tor. From the properties that we imposed on it then follows that the event {X∈ A} is in for a large collection of subsetsAof real numbers, known as theBorelsets. This class 496 A REVIEW OF PROBABILITY THEORY contains all intervals, and, roughly speaking, all sets that can be formed from intervals, by repeatedly taking unions and intersections over countable index sets. A.4 Distributions In many cases, given a random variable, we do not need the details ofΩ or the particular function X, but wish only to know thedistributionof X, which is basically a description of how likely it is thatX will take on certain values. We will concentrate on two main types of random variables. There arediscreterandom variables, which in almost all applications in this book will take nonnegative integers as values. Second, there arecontinuous random variables, which take values that vary continuously over some interval of the real line. In almost all of our applications, this interval will be [0,∞)o r[ 0 ,N] for some finiteN. We will therefore simplify the following discussion by assuming (unless otherwise mentioned) that all our random variables take nonnegative values. We will occasionally encountermixed random variables that have both a discrete and continuous part. They will be dealt with in turn as they arise. Foradiscreterandomvariable Xtherearetwomainfunctionsfordescribingthedistribu- tion.First,the probability(mass)functionf X isafunctiondefinedonthenonnegativeintegers by fX(k) = P(X = k), k= 0,1,2, …. Second, the(cumulative)distributionfunction (c.d.f.)Fis defined on [0,∞)b y FX(x) = P(X ≤ x). The two functions are related by fX(k) = FX(k)−FX(k−1), for all integers k≤ 0. FX(x) = k∑ i=0 fX(i), wherekis the greatest integer≤ x. We can define the distribution functionFX exactly as above for any random variable. The precise definition of a continuous random variable is one for which the functionF is continuous. In this case we get no information from looking at the probability thatX will take on a specific value, as this will always be zero: (P(X= r) ≤ P(X ∈ (r−h,r]) = FX(r)−FX(r−h), for allh> 0, and this goes to 0 ash→ 0). In place of the probability function, we define aprobabilitydensityfunction (p.d.f.)fX as the function satisfying P(a< X ≤ b) = ∫ b a fX(x)dx. (A.5) EXPECTATIONS AND MOMENTS 497 The two functions are related in the continuous case by fX(x) = F′ X(x), FX(t) = ∫ x 0 fX(y)dy. (A.6) (Primes denote differentiation when clear from the context.) The lower limit of 0 on the integral is a consequence of our assumption of nonnegative values the general case we would need a lower limit of−∞. When there is no confusion, we will sometimes omit subscripts and simply writef orF. The reader is cautioned that values of the density function arenotprobabilities and can take values greater than 1. We can interpret these probabilistically by the intuitive statement thatfora‘small’valueof Δx,theprobabilitythat Xwilltakeavaluebetween xandx+Δxis ‘approximately’ f(x)Δx. This can verified from (A.5). Density functions need not exist for a given continuous distribution, but throughout the book it will be assumed that they do exist, unless otherwise indicated. A.5 Expectations and moments The expectation(also known as themean) of a random variableX, represents in some sense the averagevalue that Xwill take. It is given by E(X) = ∞∑ k=1 kf(k)o r ∫ ∞ 0 xf(x)dx, depending on whetherXis discrete or continuous. Of course the above series or integral may diverge, in which case we haveE(X) =∞ . (In thegeneralcasewhere Xisnotnecessarilynonnegative-valued,theexpectationmaynotexist at all.) If g is a function defined on a set that includes the range ofX, we can define another random variableg(X) that takes the valueg(x) whenX takes the valuex. We refer to such a random variable as afunctionofX . It can be shown that the expectation of a function ofXis given by E[g(X)] = ∞∑ k=0 g(k)f(k)o r ∫ ∞ 0 g(x)f(x)dx,( A . 7 ) depending on whetherXis discrete or continuous. In particular, takingg(x) = cxfor some constantcleads to the formula E(cX) = cE(X). (A.8) Ofparticularimportancearethefunctions g(x) = xn.Forsuchafunction E[g(X)]isknown as thenth momentof X. We define thevarianceof Xby Var(X) = E[X−E(X)]2 = E(X2)−E(X)2. (A.9) 498 A REVIEW OF PROBABILITY THEORY It is clear from (A.8) that for any constantc Var(cX) = c2Var(X). (A.10) The positive square root of Var(X) is known as thestandarddeviation ofX. The smaller the variance, the more likely it is that values ofX are close to the mean. A formal statement along these lines is given byChebyshev’s inequality, which states that for k> 0, P(|X−E(X)|) ≥ k≤ Var(X) k2 . (A.11) Up to this point, we have talked first about events and probabilities, and second about random variables and expectations. It is interesting to note that, if we wish, we can subsume events and probabilities under the latter category, and speak only of random variables and expectations. For any eventAwe define a random variableI A that takes the value 1 on points in Aand the value 0 on points not inA. These are known asindicator random variables.I n thiswaytheevent Acanbeviewedasarandom variableand P(A)isequalto E[IA],ascanbe easily verified. A.6 Expectation in terms of the distribution function Itisoftendesirabletoexpressexpectationsintermsofthedistributionfunctionratherthanthe densityorprobabilityfunction.Inplaceof F,itismoreconvenienttoworkwiththefunction s(t) = 1−F(t) = P(T > t). For a simple example, consider a discrete random variableX: E(X) = f(1)+2f(2)+3f(3)+⋯ = f(1)+f(2)+f(3)+⋯ +f(2)+f(3)+⋯ +f(3)+⋯ Summing by rows, we get E(X) = ∞∑ k=0 s(k). (A.12) In the continuous case we can similarly derive E(X) = ∫ ∞ 0 s(t)dt. (A.13) JOINT DISTRIBUTIONS 499 We will in fact develop a more general formula. Supposegis a differentiable function such thatE[g(X)] exists and is finite. Noting thatf(t) =− s′(t), we integrate by parts to obtain ∫ N 0 g(t)f(t)dt=− g(N)s(N)+g(0)+∫ N 0 g′(t)s(t)dt. In the case where g(N)s(N) → 0, as N → ∞ (A.14) we can conclude that E[g(X)] = g(0)+∫ ∞ 0 g′(t)s(t)dt. (A.15) A similar derivation in the discrete case shows that, for any functiongsatisfying (A.14) for integer values ofN, E[g(X)] = g(0)+ ∞∑ k=0 [g(k+1)−g(k)]s(k). (A.16) Condition(A.14)isautomaticallysatisfiedif gisbounded,since s(N)tendsto0as Ntends to ∞,o ri fXis bounded, since thens(N) = 0 for sufficiently highN. It is also satisfied for monotoneg. Suppose, for example,gis increasing and nonnegative. Then (in the continuous case), g(N)s(N) = g(N)∫ ∞ N f(t)dt≤ ∫ ∞ N g(t)f(t)dt, and the last term must approach 0, by the hypothesis thatE[g(X)] = ∫ ∞ 0 g(t)f(t)dtis finite. A.7 Joint distributions Supposewehavetworandomvariables XandYdefinedonthesamesamplespace.Formany applications, we are interested not only in the distribution of each random variable, but also in thejointdistribution, which gives information on how the two are related. When both are discrete, we can describe this by ajointprobabilityfunction. This is a two-variable function f X,Y defined by fX,Y(k,n) = P(X = kand Y = n). (As in the single-variable case, we sometimes omit the subscripts onf when no confusion results.) WhenX and Y are both continuous we define ajointdensityfunctionf X,Y.T h i si sa 500 A REVIEW OF PROBABILITY THEORY two-variable function, such that for suitable regionsA in the plane, the probability that the point (X,Y) lies inAis given by ∫∫ A fX,Y(x,y)dxdy. (A.17) The probability or density functions associated with the component random variables are obtained from the corresponding joint function as the so-calledmarginaldistributions, namely fX(k) = ∞∑ n=0 fX,Y(k,n)o r fX(x) = ∫ ∞ 0 fX,Y(x,y)dy, (A.18) according as the random variables are both discrete or both continuous. As in the single random variable case, we assume that joint density functions exist unless otherwise indicated, but readers should be cautioned that for two continuous distributions,X, and Y, a joint density function need not exist even when bothfX and fY do. For example, if X = Y, then (X,Y) lies in the regionA= {(x,y): x= y} with probability 1, but the double integral of any two-variable function overAwill equal 0. Forafunction goftwovariablesdefinedonasetthatcontainsallthepoints (X(𝜔),Y(𝜔)) where𝜔 is in the sample spaceΩ, it can be shown, analogously to (A.7), that E[g(X,Y)] = ∞∑ k=0 ∞∑ n=0 g(k,n)fX,Y(k,n) (A.19) in the discrete case, or E[g(X,Y)] = ∫ ∞ 0 ∫ ∞ 0 g(x,y)fX,Y(x,y)dxdy (A.20) in the continuous case. An important example is the functiong(xy) = xy, which is used to define thecovarianceof Xand Y given by Cov(X,Y) = E[(X−E(X))(Y−E(Y))] = E(XY)−E(X)E(Y). (A.21) WhenCov(X,Y) > 0,itmeans thatthetworandom variables move inthesamedirection. High (low) values of one will tend to imply high (low) values of the other. Cov (X,Y) < 0 means the two random variables move in opposite directions. When Cov(X,Y) = 0, we say thatXandY are uncorrelated. For another frequently used example, takeg(x,y) =x+y. Then E(X+Y) = ∫ ∞ 0 ∫ ∞ 0 (x+y)fX,Y(x,y)dxdy= ∫ ∞ 0 ∫ ∞ 0 xfX,Y(x,y)dxdy + ∫ ∞ 0 ∫ ∞ 0 yfX,Y(x,y)dydx. CONDITIONING AND INDEPENDENCE FOR RANDOM VARIABLES 501 (Note the interchange of variables in the second integral.) Now using (A.20) we derive the often used result that E(X+Y) = E(X)+E(Y). (A.22) We need not confine ourselves to just two random variables, and may want to consider functions ofn random variables defined on the same sample space, wheren is any positive integer. In the case of a sum, formula (A.22) extends by induction to E [ n∑ i=1 Xi ] = n∑ i=1 E(Xi). (A.23) Another key formula involving a sum of random variables is Var [ n∑ i=1 Xi ] = n∑ i=1 Var(Xi)+2 ∑ i<j Cov(Xi,Xj). (A.24) A.8 Conditioning and independence for random variables The notions of independence and conditioning can be extended from events to random vari- ables. We say that two random variablesX and Y are independentif their associated events are independent. That is, for any two Borel setsAandB, P(X ∈ AandY ∈ B) = P(X ∈ A)P(Y ∈ B). Asin(A.4),wecanextendthisdefinitiontoanycollectionofrandomvariablesbyrequiring that, given any finite number of random variablesX 1,X2,…,Xn from this collection and any Borel setsA1,A2,…,An,w eh a v e P(X1 ∈ A1,X2 ∈ A2,…,Xn ∈ An) = P(X1 ∈ A1)P(X2 ∈ A2)⋯ P(Xn ∈ An). Equivalent formulations can be made in terms of the joint probability or density, or distribution functions. For example, in the bivariate case, the random variablesX and Y are independent if and only if for all points (x,y), fX,Y(x,y) = fX(x)fY(y), (A.25) or FX,Y(x,y) = FX(x)FY(y). (A.26) This leads immediately to the fact that E(XY) = E(X)E(Y)i f XandYare independent. (A.27) 502 A REVIEW OF PROBABILITY THEORY Aswewouldexpect,twoindependentrandomvariablesareuncorrelated.However, Xand Y can be uncorrelated without being independent (see Section 15.7). If X is a random variable andB an event withP(B) > 0 we can speak of the random variableX|B. This is just the restriction ofXto the sample spaceB. Then E(X|B) = the expectation ofX|B (A.28) with respect to the probability measureP(⋅|B). Conditioning is often a useful tool in computing an expectation, since we can break up the calculation into various cases. Formally we have what is known as thelaw of total expectation, which states that given a partition ofΩ, that is, a collection of pairwise disjoint setsB1,B2,…,Bn with unionΩ, E(X) = n∑ i=1 E(X|Bi)P(Bi). (A.29) TakingXto beIA for an eventAgives thelawoftotalprobability P(A) = n∑ i=1 P(A|Bi)P(Bi). (A.30) A.9 Moment generating functions Given any random variableX,t h emomentgeneratingfunction(m.g.f.) is a functionMX of a real variabletdefined by MX(t) = E[etX], (A.31) provided thatE[etX] exists in some neighbourhood of 0. (There are distributions for which E(etX) =∞ for all positive values oft, in which case the m.g.f. will not exist in any neigh- bourhood of 0.) Recalling the series expansion ex = 1+x+ x2 2! + x3 3! +⋯ , so that etX = 1+tX+ t2X2 2! + t3X3 3! +⋯ , we see that the m.g.f. has the series expansion MX(t) = 1+tE(X)+ t2E(X2) 2! + t3E(X3) 3! +⋯ , PROBABILITY GENERATING FUNCTIONS 503 from which we obtain E(Xn) = the nth derivative ofMX(t) evaluated att= 0. (A.32) Some important facts about m.g.f.’s are as follows Theorem A.1 ForX andY independent MX+Y(t) = MX(t)MY(t). Proof. Using A.26 we have MX+Y(t) = E(et(X+Y)) = E(etXetY) = E(etX)E(etY) = MX(t)MY(t), where we invoke the fact that functions of independent random variables are themselves independent to justify the third equality. The second theorem, which we state without proof, tells us that the m.g.f. determines the distribution when it exists. Theorem A.2 (The uniqueness theorem) Supposethat,fortworandomvariablesX and Y,M X(t)andMY(t)areequalinsomeneighbourhoodofzero.ThenX ∼ Y. This theorem does not say that one can easily compute the distribution function or density function from the m.g.f. The idea is rather that if one recognizes an m.g.f. as being that of a certain known distribution, then the distribution in question must be that distribution. An example of its use will appear in Section A.11.2 below. A.10 Probability generating functions For discrete distributions that take nonnegative integer values, it is often more convenient to use another function closely related to the m.g.f. This is known as theprobabilitygenerating function(p.g.f.) and is defined by P X(t) = ∞∑ k=0 tkfX(k). (A.33) This should not be confused with the probability function as defined above. Given the p.g.f., we can immediately find the complete distribution by f X(k) = P(k) X (0) k! , where the superscript (k) denotes thekth derivative with respect tot. The p.g.f. is related to the m.g.f. because we can write it as PX(t) = E(tX) = E(elogtX) = MX(logt), (A.34) 504 A REVIEW OF PROBABILITY THEORY and it follows that MX(t) = PX(et). (A.35) If we find the p.g.f. of a discrete distribution, (A.35) immediately gives us the m.g.f. as well. The following technique will illustrate a good way to remember the p.g.f.’s of familiar distributions. Suppose we start with any functiongwhose power series expansion about zero hasallnonnegativecoefficients(i.e.,allitsderivativesarenonnegativeat0)andwhoseradius of convergence is greater than 1. Then g(t) = a 0 +a1t+a2t2 +⋯ , wherean ≥ 0 for alln. Since g(1) = a0 +a1 +⋯ , such an expansion immediately gives rise to a probability distribution with probability function f(k) = a k g(1), k= 0,1, …. (A.36) The resulting p.g.f. is immediate since, for any random variableXwith this distribution, PX(t) = 1 g(1) ∞∑ k=0 aktk = g(t) g(1). (A.37) Particular applications of this idea are found in Sections A.11.2 and A.11.3 below. To calculate moments given the p.g.f., we could find the m.g.f. by (A.35) and then use (A.32), but we can also use (A.33) directly. Notice, for example, that P′ X (1) = E(X), (A.38) and similarly P′′ X (1) = E[X(X−1)], from which we can derive Var(X) = E[X(X−1)]+E(X)−E(X)2 = P′′ X (1)+P′ X (1)−P′ X (1)2. (A.39) We close this section by noting that (A.34) and (A.35) easily imply that the analogue of Theorem A.1 holds for the p.g.f. That is, ifZis an independent sum ofXandY, then PZ(t) = PX(t)PY(t). (A.40) SOME STANDARD DISTRIBUTIONS 505 A.11 Some standard distributions We review here the main features of some standard distributions. A.11.1 The binomial distribution Suppose we take m independent trials of a random experiment where the outcomes are classified as either ‘success’ or ‘failure’, and the probability of a success isp. The number of successes is a random variable with probability function p(k) = ( m k ) pk(1−p)m−k, k= 0,1, …,m. This is known as a binomial random variable with parametersmand p, and we will denote it by Bin(m,p). It can be generated, as in (A.36), by the function g(t) = (1−p+pt)m (abinomialexpansion,whichisthesourceofthename).Itfollowfrom(A.37)to(A.39)that PBin(m,p)(t) = (1−p+pt)m, E[Bin(m,p)] = mp,V a r [ B i n (m,p)] = mp(1−p). (A.41) A.11.2 The Poisson distribution The Poisson distribution arises as the limit of Binomials where the mean is held constant. Precisely, for any𝜆>0 the Poisson(𝜆) distribution is the limit of Bin(m,𝜆∕m)a smgoes to ∞. It will have a probability function given by p(k) = lim m→∞ m! k!(m−k)! ( 𝜆 m )k( 1− 𝜆 m )m−k = 𝜆k k! limm→∞ [( 1− 𝜆 m )m( 1− 𝜆 m )−k (m(m−1)(m−2)⋯ (m−k+1) mk )] . The first factor in the square brackets approaches e−𝜆, and the other two each approach 1, so we are left with p(k) = e−𝜆𝜆k k!, k= 0,1,2 …, Thisisgeneratedasin(A.36)fromthefunction g(t) = e−𝜆t sowecandeducefrom(A.37) to (A.39) that PPoisson(𝜆)(t) = e𝜆(t−1), E[Poisson(𝜆)] = 𝜆, Var[Poisson(𝜆)] = 𝜆2 +𝜆−𝜆2 = 𝜆. (A.42) 506 A REVIEW OF PROBABILITY THEORY Them.g.f.canbecalculateddirectlyfromthedefinition,butiteasiertoderiveitfrom(A.35). MPoisson(𝜆)(t) = PPoisson(𝜆)(et) = e𝜆(et−1). (A.43) Here is an example to illustrate the use of m.g.f.’s in deducting distributions. Example A.1 IfXhasaPoisson(𝜆)distributionand Y hasaPoisson( 𝜇),distribution.Find the distribution of an independent sumZ = X+Y. Solution. Directly from (A.43) and Theorem A.1 MZ(t) = MX(t)MY(t) = e(𝜆+𝜇)(et−1) and we conclude thatZhas a Poisson(𝜆+𝜇) distribution. This example illustrates a familiar phenomenon. For many common distributions, the sum of an independent collection is just another distribution of the same type but with a different parameter. In the Poisson case, since the parameter equals the mean, it is clear that the parameter for the sum of the random variables must be the sum of the parameters. A.11.3 The negative binomial and geometric distributions Let us again take repeated trials of an experiment where the probability of a ‘success’ isp. Now, however, instead of taking a fixed number of repetitions, we continue until we get a ‘failure’. We then count the number of successes. This is a random variable with p(k) = p k(1−p), k= 0,1, …. It is known as thegeometricdistribution with parameterp(the name is chosen to reflect the fact the values of the probability function form a geometric progression) and will be denoted by Geom(p). IfN is such a random variable, P(N ≥ m) = (1−p) ∞∑ k=m pk = pm. It follows that, for positive integersrand t, P(N ≥ t+r|N ≥ r) = pt+r pr = pt = P(N ≥ t). (A.44) Thissaysthatatanypoint,theprobabilitythatyouwillcontinueforafurther srepetitions is independent of the number of repetitions you have already taken, sometimes referred to as a memorylessfeature of the distribution. Suppose we repeat trials as above, except that instead of continuing until we get a single failurewecontinueuntilweget rfailures.Theprobabilityfunctionforthenumberofsuccesses SOME STANDARD DISTRIBUTIONS 507 is now given by p(k) = ( r+k−1 k ) pk(1−p)r = r(r+1)⋯ (r+k−1) k! pk(1−p)r, k= 0,1, …. (A.45) Itisclearfromthesecondexpressionthat rcanbeanypositivenumberandnotjustaninteger. Wenolongergettheinterpretationofrepeatedtrialsasgivenbutwestillhaveaperfectlyvalid distribution. It is known as thenegative binomialdistribution with parametersr and p, and we denote it by Negbin(r,p). It is generated as in (A.36) from the functiong(t) = (1−pt) −r, which is the source of the name of the distribution. From (A.37) to (A.39) we have that PNegbin(r, p)(t) = ( 1−p 1−pt )r , E[Negbin(r,p)] = rp 1−p, Var[Negbin( r,p)] = rp (1−p)2. (A.46) Theaboveformulasgivethesamequantitiesforthegeometricdistribution,simplybytaking r= 1. There is another parameterization of the negative binomial that is convenient for certain purposes. In place ofp, we use the quantity𝛼 = p∕(1−p). Then 1∕(1−p) = 1+𝛼.I nt e r m s of this parameter, we can write E[ Negbin(r,p)] = r𝛼, Var[Negbin( r,p)] = r𝛼(1+𝛼). A.11.4 The continuous uniform distribution Suppose we choose a point at random in an interval (a,b). If we decide that any interval of a givenlength hisequallylikelyregardlessofwheretheintervalstarts,thenwehaveauniform distributionU. It has a continuous density function given by fU(t) = t b−a, a< t< b, and direct integration shows that E(U) = b−a 2 ,V a r ( U) = (b−a)2 12 . (A.47) A.11.5 The normal distribution One of the most widely used distributions in probability and statistics is the normal. The standardnormal distribution is that with density function given by f(z) = 1 √ 2𝜋 e−z2∕2, −∞ < x< ∞, 508 A REVIEW OF PROBABILITY THEORY whose graph is the familiar bell-shaped curve. This is one instance where we have a random variable that takes negative values. For a standard normal random variableZ E(Z) = 0, Var(Z ) = 1, as shown by (A.32) and (A.49) below. The complete family of normal random variables consists of those of the form. 𝜇 +𝜎Z where 𝜇 is any constant𝜎> 0 and Z has a standard normal distribution. Such a random variable will have mean𝜇 and variance𝜎 2. The importance of normal distributions comes from the famouscentral limit theorem. Suppose we have an independent sequenceX1,X2,… of random variables, each with the same distribution. Then the averagesSn = (X1,X2,…,Xn)∕n converges in some sense to a normal distribution. (We will not give precise details here.) As shown below in Section A.12 it can be difficult to compute the distribution of a sum of random variables. In many cases oneinvokesthecentrallimittheoremtoapproximatethedistributionofasumbyassumingit is normal, and then only the mean and variance need be known. LetΦdenotethecumulativedistributionfunctionofthestandardnormal.Theintegration to compute this from the density function must be done by some numerical method. It used to be customary to publish tables of various values ofΦ but now it can be done by various computer programs. In Excel ®, the formula= NORMSDIST(t) returns the valueΦ(t). For example=NORMSDIST(0.5)returns0,whichisobvioussincethedistributionissymmetric aboutthemeanof0.Foranotherexample, =NORMSDIST(1.645)returns0.95,meaningthat with probability 0.95 a standard normal takes a value less than or equal to 1.645. In most applications we are interested in the inverse calculation. Given𝛼 we we want to find the𝛼-quantile of the distribution, that is the pointt so that the probability of being less than t is 𝛼.T h i si sΦ−1(𝛼), which in Excel® is denoted by NORMSINV(𝛼). For example, taking𝛼 = 0.95, the formula=NORMSINV(0.95) returns 1.645, as we have verified. For the case of a general normalX = 𝜇 +𝜎(Z)w eh a v e X ≤ x if and only if 𝜇 +𝜎(Z) ≤ x if and only if Z ≤ (x−𝜇)∕𝜎, from which it follows that FX(x) =Φ (x−𝜇 𝜎 ) , fX(x) = 1 𝜎fZ (x−𝜇 𝜎 ) (A.48) where we invoke (A.6) for the second equation. It follows moreover that the𝛼-quantile ofX is given by𝜇 +Φ−1(𝛼)𝜎. To calculate the m.g.f. we start withZ, the standard normal. Then MZ(t) = 1√ 2𝜋∫ ∞ −∞ etxe−x2∕2dx. SOME STANDARD DISTRIBUTIONS 509 Wehandlethisbythefamiliar‘completingthesquare’trick.Theexponentintheintegrand is −1 2[x2 −2tx+t2 −t2] =− 1 2[(x−t)2 −t2]. Sincex is the variable of integration here, the terms involvingtcan be factored out, and we write MZ(t) = et2∕2 [ 1√ 2𝜋 ∫ ∞ −∞ e−(x−t)2∕2dx ] . Now the term inside the bracket is just the integral over the entire range of the density function of the normal distribution with meant and standard deviation 1, as we see from (A.48) so it must equal 1. We conclude that MZ(t) = et2∕2. (A.49) To handle the general case, we will use some formulas involving modifications by con- stants. For any constanta, MX+a(t) = E[et(X+a)] = etaE[etX] = etaMX(t). (A.50) For any constantb, MbX(t) = E[etbX] = MX(bt). (A.51) Thegeneralnormalrandomvariable Xwithmean 𝜇andstandarddeviation 𝜎isdistributed as 𝜇 +𝜎Zthat by combining (A.49)–(A.51), we obtain MX(t) = et𝜇+𝜎2t2∕2. (A.52) A.11.6 The gamma and exponential distributions The gamma distribution is produced from the gamma function, defined by Γ(𝛼) = ∫ ∞ 0 x𝛼−1e−xdx, 𝛼> 0. Wecannotevaluatethisintegralintermsofelementaryfunctions.Wecan,however,getsome information by integrating by parts. For𝛼> 1, Γ(𝛼) =− x 𝛼−1 e−x| | ∞ 0 +(𝛼 −1)∫ ∞ 0 x𝛼−2e−xdx= (𝛼 −1)Γ(𝛼 −1). (A.53) Since Γ(1) = 1, it follows by induction that for any positive integern, Γ(n) = (n−1)!. 510 A REVIEW OF PROBABILITY THEORY Choose any𝛽> 0 and consider a more general integral, ∫ ∞ 0 x𝛼−1 e−𝛽xdx= Γ(𝛼) 𝛽𝛼 , whichfollowsbymakingachangeofvariablefrom xto𝛽x.Fromthis,wedefineacontinuous positive-valued random variable with the density function f(x) = 𝛽𝛼 Γ(𝛼)x𝛼−1e−𝛽x, x> 0. This is known as a Gamma distribution with parameters𝛼 and 𝛽 and will be denoted by Gamma(𝛼,𝛽). For such a distributionX MX(t) = 𝛽𝛼 Γ(𝛼) ∫ ∞ 0 x𝛼−1 e−(𝛽−t)xdx= 𝛽𝛼 Γ(𝛼) Γ(𝛼) (𝛽 −t)𝛼 = ( 𝛽 𝛽 −t )𝛼 . (A.54) Evaluating derivatives at 0 then gives E(X) = 𝛼 𝛽, E(X2) = 𝛼(𝛼 +1) 𝛽2 ,V a r ( X) = 𝛼 𝛽2. (A.55) Thespecialcaseofthegammawhen 𝛼 = 1arisesfrequently.Itisknownasan exponential distributionandwillbedenotedbyExp(𝛽 ).Forsuchadistribution Xwehavefromabovethat fX(t) = 𝛽e−𝛽t, sX(t) = e−𝛽t, MX(t) = 𝛽 𝛽 −t, E(X) = 1 𝛽,V a r ( X) = 1 𝛽2. (A.56) Thisistheuniquecontinuousdistributionwhichhasthesame“memoryless”propertythat we observed in the discrete case for the geometric distribution, namely, for any nonegativet, andr P(X > t+r|X > r) = P(X > t). (A.57) A.11.7 The lognormal distribution Another commonly used positive-valued distribution is thelognormal. A random variableY is said to have a lognormal distribution ifY = eX where Xhas a normal distribution. That is, justasthenamesuggests,thelogarithmof Y isnormal.Wecancalculatethemomentsofthis distribution from (A.52). Namely, ifX ∼ N(𝜇,𝜎2) (the normal distribution with mean𝜇 and variance𝜎2), then E(Y) = E(eX) = MX(1) = e𝜇+𝜎2∕2, (A.58) and E(Y2) = MX(2) = e2𝜇+2𝜎2 , (A.59) CONVOLUTION 511 from which we obtain Var(Y) = E(Y2)−(EY)2 = e2(𝜇+𝜎2)(e𝜎2 −1). A.11.8 The Pareto distribution The probability density function of the Pareto (𝜃,𝛼) distribution is proportional to (x+ 𝜃)−(𝛼+1). Since ∫ ∞ 0 (x+𝜃)−(𝛼+1) dx= 𝜃−𝛼 𝛼 , this density function is given by f(x) = 𝛼𝜃𝛼 (x+𝜃)𝛼+1, For this distribution, s(x) = 𝛼𝜃𝛼 ∫ ∞ x (y+𝜃)−(𝛼+1) dy= ( 𝜃 x+𝜃 )𝛼 and, provided𝛼> 1, E(X) = ∫ ∞ 0 s(t)dt= 𝜃 𝛼 −1. Further integration leads to E(X2) = 2𝜃2 (𝛼 −1)(𝛼 −2), provided 𝛼> 2. In general,E(Xk) becomes infinite fork≥ 𝛼. A.12 Convolution A.12.1 The discrete case Convolution is a tool for finding the distribution of a sum of random variables. Consider a simple example. SupposeX takes the values 2, 5 and 8 with probabilities 0.5, 0.3 and 0.2, respectively, whileY takes the values 3, 6 and 9, with probabilities 0.6, 0.3 and 0.1, respectively, and thatX and Y are independent. LetZ = X+Y. Thiscanhappenifandonlyif X = 2and Y = 3.Bytheindependenceassumption,we multiply to get the probability of both these occurrences, and we see that f Z(5) = 0.5×0.6 = 0.30. 512 A REVIEW OF PROBABILITY THEORY Whatistheprobabilitythat Z = 8?Therearetwomutuallyexclusivewaysthatthiscanoccur, namely,X = 2,Y = 6 andX = 5,Y = 3. We deduce that fZ(8) = fX(2)fY(6)+fX(5)fY(3) = 0.5×0.3+0.3×0.6 = 0.33. For the probability thatZ = 11, we must take three terms: fZ(11) = fX(2)fY(9)+fX(5)fy(6)+fX(8)fY(3) = 0.5×0.1+0.3×0.3+0.2×0.6 = 0.26. Proceeding similarly, we can calculatefZ(14) = 0.09 andfZ(17) = 0.02, completing the dis- tribution. Thegeneralrulecanbewrittenasfollows.Let XandYbeindependentrandomvariables, with nonnegative integers as values, and letZ = X+Y. Then fZ(n) = ∑ fX(k)fY(m), (A.60) where the sum is taken over all pairs (k,m)f o rw h i c hk+m= n. Formula(A.60)isfineforsmallexampleslikethis,butincaseswheretherandomvariables take on many more values (perhaps even an infinite number) we want to write the formula in a more systematic way, which will ensure that we do not miss any combinations. In order for a sum of nonnegative integersk+m to sum to a nonnegative integern, we can havek take any value from 0 tonand thenmmust take the valuen−k. We can then write (A.60) as f Z(n) = n∑ k=0 fX(k)fY(n−k). (A.61) Equation (A.61) is the general so-called convolution formula. Note, however, that for small problems like the one above, which we want to do by hand calculation, it can be inefficient compared to (A.60) since we will be adding up many terms of 0. Suppose we want the probability thatZ ≤ n. We can reason in the same way. We needX to take a valuekfrom 0 tonand nowY must take a value less than or equal ton−k,s ow e have F Z(n) = n∑ k=0 fX(k)FY(n−k). (A.62) It is not hard to verify that we can interchangeFandf also write (A.28) as FZ(n) = n∑ k=0 FX(k)fY(n−k). (A.63) This follows simply by changing the variable of summation fromk to n−k. The reader is warned that wecannotwriteFZ(n) = ∑n k=0 FX(k)FY(n−k). Asanillustrateofconvolution,wewillredoExampleA.1,whichwedidbeforebyp.g.f.’s, now using convolution. CONVOLUTION 513 From (A.61), fZ(n) = n∑ k=0 e−𝜆 e−𝜇 𝜆k𝜇n−k k!(n−k)! = e−(𝜆+𝜇)(𝜆+𝜇)n n! n∑ k=0 ( n k )( 𝜆 𝜆+𝜇 )k( 𝜇 𝜆+𝜇 )n−k . The summation on the right hand side is just a binomial expansion [( 𝜆 𝜆+𝜇 ) + ( 𝜇 𝜆+𝜇 )]n = 1, and we conclude thatZhas a Poisson(𝜆+𝜇) distribution. Supposewewantthesumofmorethantwodiscreterandomvariables.Forexample,given independent random variablesX,Y,W, find the distribution ofV = X+Y+W. For cases where there are few nonzero values, we can proceed just as in (A.60) and write fV(n) = ∑ fX(k)fY(m)fW(p), where the sum is taken over all ordered triples (k,m,p) such thatk+m+p= n.F o rt h e general case, however, it may not be so easy to pick out all such triples, and we have to proceed more systematically. The basic procedure is to iterate the calculation. We first apply (A.60) or (A.61) to find the distribution ofZ = X+Y and then apply it again to find the distribution ofV = Z+W. For the general case of a sum ofnindependent random variables, we just iterate thisn−1 times. Example A.2 Let Z = ∑ 5 i=1 Xi, where theXi’s are independent random variables, each takingthevalue0withprobability0.7,1withprobability0.2and2withprobability0.1.Find the probability thatZ = 2. Solution. We can derive this single number by inspection, without having to do the four iterations. One way for the five values to add up to 2 is that one of them is 2 and the rest are zero. Since there are five possibilities for the 2, the probability of this is 5×0.7 4 ×0.1. The only other way is for three of the values to be zero and the other two to be 1. The probability of this 10×0.7 3 ×0.22.S o P(Z = 2) = 5×0.74 ×0.1+10×0.73 ×0.22 = 0.2573. A.12.2 The continuous case We now suppose thatX and Y are independent,continuous, nonnegative random variables, and we want the distribution ofZ = X+Y.I fXtakes the valuexandY takes the valuey,t h e sum will be less than or equal tosif and only if the point (x,y) lies in the region bounded by 514 A REVIEW OF PROBABILITY THEORY thexandyaxes, and the liney= s−x. Integrating the joint density function over this region gives FZ(s) = ∫ s 0 ∫ s−x 0 fX,Y(x,y)dydx. (A.64) Equation(A.64)isinfacttruewithouttheindependenceassumption,butwiththeassumption of independence the integral simplifies considerably. The joint density function factors as f X(x)fY(y), the inner integral becomes ∫ s−x 0 fY(y)dy= FY(s−x) and we can write FZ(s) = ∫ s 0 fX(x)FY(s−x)dx, (A.65) a direct analogue of (A.62). Using Leibniz’s rule for differentiating integrals and the fact thatFY(0) = 0, we have fZ(s) = ∫ s 0 fX(x)fY(s−x)dx, (A.66) a direct analogue of (A.61). The following example serves to indicate that calculating convolutions for continuous distributions can be a very involved procedure, even in the simplest cases. Example A.3 Find the distribution of an independent sumZ = X+Y, where X has a uniform distribution on the interval [0, 2] andY has a uniform distribution on the interval [0, 3]. Solution. Whether we use (A.65) or (A.66) depends on the particular example. In this case, it is easier to use (A.66). See Figure A.1, and notice that it depicts a region of the (x,s) plane (rather than the (x,y) place. It is that portion of g the positive quadrant in the (x,s) plane givenby0 ≤ s≤ 5,0 ≤ x≤ s.Adistributionthatisuniformonaninterval[ a,b]hasadensity function that is a constant (b−a) −1 on this interval and zero elsewhere. Consequently, the integrandin(A.66)takesthevalue1/6when0 ≤ x≤ 2and0 ≤ s−x≤ 3,asindicatedbythe union of regionsR1,R2,R3, and it take the value of 0 elsewhere, as indicated by the union of regionsS1,S2,S3. Now we consider in turn all possible values ofs and the value of the integral in (A.66). CONVOLUTION 515 S1 R1 R2 R3 S2S3 x s s = 2 s = 3 s = 5 x = 2 x = 3 x = 50 Figure A.1 The region of integration in Example A.3 If0 ≤ s≤ 2,thevalueof xinR1 willvaryfrom0to s,sothattheintegrandwillbe1/6for allx. Therefore, fZ(s) = s 6,0 < s≤ 2. If 2≤ s≤ 3, the value ofxin R2 will vary from 0 to 2, so that the integrand will be 1/6 whenx≤ 2 and zero when 2< x≤ s. Therefore, fZ(s) = 1 3,2 ≤ s≤ 3. If 3≤ s≤ 5, then the value ofXinR3 will vary froms−3 to 2, so that the integrand will be 1/6 whens−3 ≤ x≤ 2, but zero when 0≤ x< s−3 or when 2≤ x. Therefore, fZ(s) = 5−s 6 ,3 ≤ s< 5. Of course, sinceZvaries from 0 to 5, we know that fZ(s) = 0, s< 0o rs> 5. A.12.3 Notation and remarks Suppose thatX has probability (or density) functionf, and distribution functionF, whileY has probability (or density) functiongand distribution functionG. The probability (or density) function given in (A.61) (or (A.66)) is denoted byf ∗ g. The distribution function given in (A.62) (or (A.65)) is denoted byF ∗ G.O n em u s tt a k e care with this latter notation as it could induce the error that the reader was warned about 516 A REVIEW OF PROBABILITY THEORY above, of using the c.d.f. in both places when calculating the c.d.f. of the sum. Similarly, given a sum ofnindependent random variables with the same distribution, we usef ∗n for the probability (density) function of the sum andF∗n for the distribution function of the sum. To illustrate, in Example A.2 we would denoted the desired answer asf∗5(2), wheref was the given probability function. As a final remark in this section, we note that all of the above formulas hold for random variablesthattakenegativevalues,withtheexceptionthatwemustreplacethelowerlimiton the integrals and sums by−∞. A.13 Mixtures Suppose you are faced with a choice of two games to play. Game 1 has a return of either 2o r−2 each with probability 1/2, while game 2 has a return of 4 with probability 1/4, 2 with probability 1/4 or−3 with probability 1/2. Racked with indecision, you flip a coin, intendingtopaygame1ifaheadturnsup,orgame2ifatailscomes up.Whatyouarereally playing is amixtureof game 1 and game 2 with equal weights. The resulting return is easily calculatedtobe4withprobability1/8,2withprobability3/8, −2withprobability2/8and −3 with probability 2/8. More generally, we could havenrandom variablesX 1,X2,…,Xn and a probability distribution on {1,2,…,n}, called the mixing distribution. The resulting mixture isarandomvariable Xthattakesavaluefrom Xiwithprobability p(i),where pistheprobability functionofthemixingdistribution.(Ourinitialexample had p(1) = p(2) = 1∕2.)Calculating quantities with respect to the mixed distribution presents no problems as everything follows the same convex combination. We have, for example, f X(s) = n∑ k=1 p(k)fXk(s), FX(s) = n∑ k=1 p(k)FXk(s), MX(s) = n∑ k=1 p(k)MXk(s). (A.67) Letusconsideraslightlytrickiersituation,inwhichwehavecontinuousmixing.Suppose now that we have a whole family of random variablesXt, either all discrete or all continuous, indexed on all the nonnegative reals [0,∞) instead of just the integers. For a mixing distribu- tion,wetakeacontinuousnonnegativerandomvariablewithdensityfunction p.Theresulting mixture is a random variableXwith density function fX(s) = ∫ ∞ 0 p(t)fXt(s)dt. (A.68) References Arrow, K.J. (1963). Uncertainty and the welfare of medical care.American Economic Review53, 941–973. Artzner, P., Delbaen, F., Eber, J.M. and Heath, D. (1999). Coherent measures of risk.Math Finance9, 203–228. Bj¨ork, T. (2009).Arbitrage Theory in Continuous Time, 3rd edn. New York, NY: Oxford University Press. Bowers,N.,Gerber,H.,Hickman,J.,Jones,D.andNesbitt,C.(1997). Actuarial Mathematics,2ndedn. Schaumburg, IL: Society of Actuaries. Brillinger, D.R. (1961). A justification of some common laws of mortality.Transactions of the Society of ActuariesXIII, 116–119. Broverman, S. (2010).Mathematics of Investment and Credit, 5th edn. Winstead CT: ACTEX. B¨ulhman,H.andGisler,A.(2005). A Course in Credibility Theory.Berlin-Heidelberg:Springer-Verlag. Carri`ere, J. (1994a). An investigation of the Gompertz law of mortality.Actuarial Research Clearing House,2 . Carri`ere,J.(1994b).Dependentdecrementtheory. Transactions of the Society of ActuariesXLVI,45–65. Daniel,J.W.andVaaler,L.J.F.(2009). Mathematical Interest Theory,2ndedn.MathematicalAssociation of America. Dickson, D., Hardy, M. and Waters, H. (2013).Actuarial Mathematics for Life Contingent Risks, Cambridge: Cambridge University Press. Etheridge, A. (2002).A Course in Financial Calculus, Cambridge: Cambridge University Press. Frees, E., Carri`ere, J. and Valdez, E. (1996). Annuity valuation with dependent mortality.Journal of Risk and Insurance63, 229–261. Frees, E. and Valdez E. (1998). Understanding relationships using copulas.North American Actuarial Journal 2, 1–25. Gerber,H.andShiu,E.S.(1998).Onthetimevalueofruin. North American Actuarial Journal2,48–78. Grandel, J. (1991).Aspects of Risk Theory. New York, NY: Springer-Verlag. Herzog, T.N. (1999).Introduction to Credibility Theory, 3rd edn. Winstead, CT: ACTEX. Hoel,P.,Port,S.andStone,C.(1972). Introduction to Stochastic Processes,Boston:HoughtonMifflin. Fundamentals of Actuarial Mathematics, Third Edition. S. David Promislow. © 2015 John Wiley & Sons, Ltd. Published 2015 by John Wiley & Sons, Ltd. Companion website: http://www.wiley.com/go/actuarial 518 REFERENCES Hull,J.C.(2014). Options, Futures, and other Derivative Securities,9thedn,Cambridge,UpperSaddle River, NJ: Prentice Hall. Jones, B.L. (1997). Stochastic models for continuing care retirement communities.North American Actuarial Journal1, 50–73. Jordan, C.W. (1967).Life Contingencies, 2nd edn. Schaumburg, IL: Society of Actuaries. Kaas, R., Goovaerts, Dhaene, M.J. and Denuit, M. (2008).Modern Actuarial Risk Theory, 2nd edn. Springer-Verlag. Kemeny, J.G. and Snell, J.L. (1963).Finite Markov Chains. Princeton, NJ: D. Van Nostrand. Klugman, S., Panjer, H. and Willmot, G. (2012).Loss Models, 4th edn. Hoboken, NJ: John Wiley & Sons. London, D. (1997).Survival Models and their Estimation, 3rd edn. Winstead CT: ACTEX. McDonald, R.L. (2012).Derivatives Markets, 3rd edn. Upper Saddle River, NJ: Prentice Hall. Mikosch, T. (1998).Elementary Stochastic Calculus-with Finance in View,R i v e rE d g e ,N J :W o r l d Scientific. Nelsen, R.B. (1999).An Introduction to Copulas, Lecture Notes in Mathematics 139. New York, NY: Springer-Verlag. Norberg, R. (2008). Multi-state models for life insurance mathematics. In: E. Melnick and B. Everitt (eds.), Encyclopedia of Quantitative Risk Analysis and Assessment. Chichester: Wiley-Blackwell. Powers, M.R. (1995). A theory of risk, return, and solvency.Insurance: Mathematics and Economics 17 , 101–118. Promislow,S.D.(1980).Anewapproachtothetheoryofinterest. Transactions of the Society of Actuaries XXXII, 53–92. Promislow, S.D. (1991). Select and ultimate models in multiple decrement theory.Transactions of the Society of ActuariesXLIII, 281–300. Promislow, S.D. (1997). Classification of usurious loans. In: M. Sherris (ed.),Proceedings of the 7th International AFIR Colloquium. Institute of Actuaries of Australia. Ross, S.M. (2010).Introduction to Probability Models, 10th edn. San Diego, CA: Academic Press. Rothschild, M. and Stiglitz, J. (1970). Increasing risk: A definition.Journal of Economic Theory2, 225–243. Royden, H.L.(1988).Real Analysis, 3rd edn. New York, NY: Macmillan. Rudin, W.R. (1976).Principles of Mathematical Analysis, 3rd edn. New York, NY: McGraw-Hill. Shaked, M. and Shantikumar, J. G. (2007).Stochastic Orders. Springer. Steland, A. (2012).Financial Statistics and Mathematical Finance, Chichester: John Wiley & Sons. Teichroew, D., Robichek, A.A. and Montalbano, M. (1965a). Mathematical analysis of rates of return under certainty.Management Science11, 395–403. Teichroew, D., Robichek, A.A. and Montalbano, M. (1965b). An analysis of criteria for investment and financing decisions under certainty.Management Science12, 151–179. Tennenbein, A. and Vanderhoof, I. (1980). New mathematical laws of select and ultimate mortality. Transactions of the Society of ActuariesXLIV, 509–538. Young,V.R.(2004).Premiumprinciples.In:J.F.TeugelsandB.Sundt(eds.), Encyclopedia of Actuarial Science. Chichester: John Wiley & Sons. Notation index The following is a list of the major symbols which are used in the book. For the most part, the first page they appear on is listed. An exception is the notation in Appendix A, where the first appearance in that chapter is noted. This list excludes that part of the standard actuarial notation which is not used in the main body of the text. The latter can be found in the appropriatesectionsofChapters2–6,8and10entitled‘StandardNotationandTerminology’. Chapter 2 ̈a(c;v)1 5 a ∗ b B k(c;v)2 1 ̃Bk(c)2 5 kc 21 kc 21 c ◦ k 25 dk 12 ek 17 ik 12 kV(c;v)2 1 v(k)1 1 v(k,n)1 0 v ◦ k 25 Valn(c;v)2 6 Δb 17 ∇b 18 Chapter 3 d x 39 ex 42 e◦ x 42 npx 40 nqx 40 px 40 qx 40 𝓁x 39 𝜔 40 Chapter 4 ̈ax(c)4 9 (1∞)5 1 yx(k)5 0 ̈a[x]+k(c)5 2 Chapter 5 A x(b)6 2 wx(k)6 9 ̈A[x]+k(c)7 0 Chapter 6 𝜂 k 81 kV 78 Fundamentals of Actuarial Mathematics, Third Edition. S. David Promislow. © 2015 John Wiley & Sons, Ltd. Published 2015 by John Wiley & Sons, Ltd. Companion website: http://www.wiley.com/go/actuarial 520 NOTATION INDEX Chapter 7 ̈a(m)(c;y)9 9 a(m)(c;y) 104 ̈a(m) x (c) 101 d(m) 100 i(m) 100 𝛼(m) 102 𝛽(m) 102 Chapter 8 ̄a(c;y) 113 ̄ax(c) 115 ̄Ax(b) 119 𝛼(∞) 116 𝛽(∞) 116 𝛿y(t) 113 𝛿 114 𝜆x(t) 119 𝜇(x) 118 𝜇x(t) 118 Chapter 9 𝓁 [x]+k 139 sp[x]+t 138 sq[x]+t 137 q[x]+t 138 Chapter 10 ̈axy(c) 145 ̄axy(c) 145 ̈axy(c) 147 Axy(c) 148 ̄Axy(c) Axy(c) ̄Axy(c) 148 A1 xy (b) 152 A2 xy (b) 152 ̄A1 xy (b) 153 ̄A2 xy (b) 153 npxy 143 npxy 146 nqxy 146 nqxy 153 sq1 xy 147 𝜇xy(t) 147 Chapter 11 ̄A(j) x (b) 166 d(j) x 163 d(𝜏) x 163 𝓁(𝜏) x 163 𝓁(j) x 164 np(j) x 165 np(𝜏) x 160 nq(j) x 164 q(j) x 164 q′(j) x 171 𝜇(j) x (t) 165 Chapter 12 AS k 187 Prk 190 Πk 194 Chapter 13 AV k 201 COIk 202 Sx 205 Chapter 14 f x(t) 220 sx(t) 220 T◦u 216 ̃T 217 Chapter 15 ̈ãT (c;v) 229 ̄aT (c,v) 231 ÃT (b,v) 225 ̄AT (b;v) 226 CV(X) 238 rL 234 L 235 Chapter 16 ̂FX 252 ̂fX 252 Chapter 17 f T1,T2,…,TM (t1,t2,…,tm) 260 FT1,T2,…,TM (t1,t2,…,tm) 260 NOTATION INDEX 521 sT1,T2,…,TM (t1,t2,…,tm) 260 FT,J(t,j) 262 fT,J(t,j) 264 sT,J(t,j) 264 𝜇T,J(t,j) 266 Chapter 18 p xy(k,n) 284 pxy 284 o(h) 294 ∼ 284 Chapter 19 p ii(s,t) 311 pii(s,t) 314 𝜇ij(t) 312 Chapter 20 E k(W) 350 EΠ(W) 366 Chapter 21 ⟨N,X⟩ 379 S + 384 (X −d)+ 388 X ∧d 388 Chapter 22 VaR 413 TVaR 𝛼 413 Chapter 23 D(u) 423 J(u) 433  440 𝜓(u) 420 𝜓 k(u) 436 𝜓(u,t) 420 Chapter 24 E(X|Y) 453 Var(X|Y) 455 ∝ 459 Appendix A P(A) 477 P(A|B) 479 f X(x) 480 FX(x) 480 E(X) 481 Var(X) 481 s(t) 482 Cov(X,Y) 484 E(X|B) 486 f ∗ g 499 F ∗ G 499 f ∗n 499 F∗n 499 MX(t) 486 PX(t) 487

Human: What is the probability that
Z = 5?

Actuarial AI: Appendix A review of probability theory Thisappendixprovidesareviewofthebasicprobabilitytheoryusedinthisbook.Inaddition, more specialized topics in probability theory, will appear in the relevant chapters as they are needed. It is expected that most readers will be at least somewhat familiar with this material, so the pace is fairly rapid, with few examples or derivations. A.1 Sample spaces and probability measures We model the results of a random experiment by a setΩ, known as asample space, where each point ofΩ corresponds to a possible outcome. For example, if we throw a pair of dice, thesamplespacecouldconsistofthe36orderedpairs( a,b)where aandbtakevaluesfrom1 to6.Acombinationofoutcomes,knownasan event,isrepresentedbyasubsetof Ω.Suchan event occurs if any of the outcomes in the subset occur. In the above example, the event that thetotalonthediceis10,wouldberepresentedbytheset{(4,6),(5,5),(6,4)}.Fromagiven collection of events, familiar set operations can be applied to build other events. The union of events, denoted with the symbol∪, gives us the event that occurs if any one of the given events occur. The intersection of events, denoted with the symbol∩, gives us the event that occurs if all of the given events occurs. The complement of an eventA, denoted byA c,g i v e s us the event that occurs ifAdoes notoccur. Two events are said to bemutually exclusiveif the occurrence of one means that the other cannot occur. These are represented by subsetsA andBthat aredisjoint, that is,A∩B=∅ (the empty set). To each eventAwe assign a numberP(A) in the interval [0,1], known as theprobability ofA.Thismeasureshowlikelyitisfortheevent Atooccurinasingletrialoftheexperiment. Theassignment P(A)mustsatisfythefollowingfundamentalrule:Givenafiniteorcountably FundamentalsofActuarialMathematics , Third Edition. S. David Promislow. © 2015 John Wiley & Sons, Ltd. Published 2015 by John Wiley & Sons, Ltd. Companion website: http://www.wiley.com/go/actuarial 494 A REVIEW OF PROBABILITY THEORY infinite sequence A1,A2,…,An,… of pairwise disjoint events – that is,Ai ∩Aj =∅ for i≠ j– then P [ ⋃ i Ai ] = ∑ i P(Ai). (A.1) Moreover, we require thatP(Ω) = 1. These requirements can be motivated by adopting the relative frequency interpretation of P, wherebyP(A) gives the expected proportion of times that the eventA will occur if the experiment is repeated a sufficiently large number of times. We often refer toPas aprobabilitymeasure on Ω. The following are some simple, frequently used consequences of (A.1), P(∅) = 0, A⊆ BimpliesP(A) ≤ P(B), P(Ac) = 1−P(A), (A.2) and P(A∪B) = P(A)+P(B)−P(A∩B). (A.3) We now discuss a technical difficulty that arises when the sample space is not finite or countably infinite. In this case, postulating thatP(A) should be defined for all subsetsA is overly restrictive, and would prevent one from finding suitable probability measures in many cases.Accordinglyweonlyrequirethat P(A)bedefinedwhen Abelongstoacertainspecified collectionofsubsets,whichwedenoteby .Wedo,however,requirecertainrestrictionson  toensurethatcountablefamiliesofeventscanbecombinedinthefamiliarwayswedescribed above. These requirements are as follows: (a) Given any finite or countably infinite sequenceA 1,A2,…,An,… of sets in, (i) their union is in, (ii) their intersection is in. (b) For anyA∈ , the complementAc ∈ . (c) The whole setΩ∈ . A collection of subsets with these properties is known as a𝜎-field. (Since (a)(ii) follows from (a)(i) and (b), while (a)(i) follows from (a)(ii) and (b), many authors omit one of (a)(i) or (a)(ii) in the definition of a𝜎-field.) As a consequence of (a)(i) we know that the left hand side of (A.1) makes sense. To summarize, in modelling a random experiment, we choose a sample spaceΩ, a prob- ability measurePand a𝜎-field . WhenΩ is finite or countably infinite, the collection is invariably taken to be all subsets. In the remaining sections of this appendix, we assume a fixedΩ,Pand. RANDOM VARIABLES 495 A.2 Conditioning and independence Ifwearetoldthatacertainevent Bhasoccurred,weneedtoreassessprobabilities.Assuming P(B) > 0, this is done by defining a new probability measure, denoted byP(⋅|B), as follows: P(A|B) = P(A∩B) P(B) , A∈ . Weoftenrefertothelefthandsideas‘theprobabilityof AgivenB’.Thisnewmeasureassigns probability0toeventsdisjointfrom B,andfor A⊆ Bitassignstheprobability P(A)∕P(B).In a sense, we can view this as changing the sample space fromΩ to B, and then dividing by a constant to ensure that the probability of the entire space is 1. The symbolP(A|B1,B2,…,Bn) denotes the probability ofAgiven that all of the events Bi have occurred. That is, it equalsP(A|B) whereB=∩ n i=1 Bi. We say thatAandBare independentif P(A|B) = P(A). In other words, knowing thatB occurred does not affect our original assessment of the likelihood ofA. It is convenient to write this condition in the symmetric form P(A∩B) = P(A)P(B), whichshowsimmediatelythatwealsohave P(B|A) = P(B).Moreover,thisformmakessense if either set has probability 0, in which case we automatically have independence. More generally, we define a collection of events (possibly infinite) to be independent if, given any finite subcollectionA1,A2,…,An, P [ n⋂ 1=i Ai ] = P(A1)P(A2)⋯ P(An). (A.4) A.3 Random variables Randomvariables are,intuitively,numericalquantitiesassociatedwitharandomexperiment. For example, we toss 100 coins and count the number of heads. Formally, we represent the random variable by a real-valued functionX defined onΩ, where for𝜔 ∈Ω , X(𝜔)i s the number obtained when the outcome of the experiment is𝜔. The technical restriction we referred to above whenΩ is not finite or countably infinite also puts restrictions on the possible functions we consider. Given a random variable, traditionally denoted by a capital letterlike X,andasubset Aofrealnumbers,wewouldliketofindtheprobabilitythat Xtakes a value in the setA. This is not possible in our model if the event {𝜔∈Ω : X(𝜔) ∈ A} is not in the collection of events that we are able to compute probabilities for. Accordingly, in the definition of a random variable we postulate that at least forA of the form (−∞,r], the corresponding event is in, so we can always compute the probability thatX is less than or equal tor. From the properties that we imposed on it then follows that the event {X∈ A} is in for a large collection of subsetsAof real numbers, known as theBorelsets. This class 496 A REVIEW OF PROBABILITY THEORY contains all intervals, and, roughly speaking, all sets that can be formed from intervals, by repeatedly taking unions and intersections over countable index sets. A.4 Distributions In many cases, given a random variable, we do not need the details ofΩ or the particular function X, but wish only to know thedistributionof X, which is basically a description of how likely it is thatX will take on certain values. We will concentrate on two main types of random variables. There arediscreterandom variables, which in almost all applications in this book will take nonnegative integers as values. Second, there arecontinuous random variables, which take values that vary continuously over some interval of the real line. In almost all of our applications, this interval will be [0,∞)o r[ 0 ,N] for some finiteN. We will therefore simplify the following discussion by assuming (unless otherwise mentioned) that all our random variables take nonnegative values. We will occasionally encountermixed random variables that have both a discrete and continuous part. They will be dealt with in turn as they arise. Foradiscreterandomvariable Xtherearetwomainfunctionsfordescribingthedistribu- tion.First,the probability(mass)functionf X isafunctiondefinedonthenonnegativeintegers by fX(k) = P(X = k), k= 0,1,2, …. Second, the(cumulative)distributionfunction (c.d.f.)Fis defined on [0,∞)b y FX(x) = P(X ≤ x). The two functions are related by fX(k) = FX(k)−FX(k−1), for all integers k≤ 0. FX(x) = k∑ i=0 fX(i), wherekis the greatest integer≤ x. We can define the distribution functionFX exactly as above for any random variable. The precise definition of a continuous random variable is one for which the functionF is continuous. In this case we get no information from looking at the probability thatX will take on a specific value, as this will always be zero: (P(X= r) ≤ P(X ∈ (r−h,r]) = FX(r)−FX(r−h), for allh> 0, and this goes to 0 ash→ 0). In place of the probability function, we define aprobabilitydensityfunction (p.d.f.)fX as the function satisfying P(a< X ≤ b) = ∫ b a fX(x)dx. (A.5) EXPECTATIONS AND MOMENTS 497 The two functions are related in the continuous case by fX(x) = F′ X(x), FX(t) = ∫ x 0 fX(y)dy. (A.6) (Primes denote differentiation when clear from the context.) The lower limit of 0 on the integral is a consequence of our assumption of nonnegative values the general case we would need a lower limit of−∞. When there is no confusion, we will sometimes omit subscripts and simply writef orF. The reader is cautioned that values of the density function arenotprobabilities and can take values greater than 1. We can interpret these probabilistically by the intuitive statement thatfora‘small’valueof Δx,theprobabilitythat Xwilltakeavaluebetween xandx+Δxis ‘approximately’ f(x)Δx. This can verified from (A.5). Density functions need not exist for a given continuous distribution, but throughout the book it will be assumed that they do exist, unless otherwise indicated. A.5 Expectations and moments The expectation(also known as themean) of a random variableX, represents in some sense the averagevalue that Xwill take. It is given by E(X) = ∞∑ k=1 kf(k)o r ∫ ∞ 0 xf(x)dx, depending on whetherXis discrete or continuous. Of course the above series or integral may diverge, in which case we haveE(X) =∞ . (In thegeneralcasewhere Xisnotnecessarilynonnegative-valued,theexpectationmaynotexist at all.) If g is a function defined on a set that includes the range ofX, we can define another random variableg(X) that takes the valueg(x) whenX takes the valuex. We refer to such a random variable as afunctionofX . It can be shown that the expectation of a function ofXis given by E[g(X)] = ∞∑ k=0 g(k)f(k)o r ∫ ∞ 0 g(x)f(x)dx,( A . 7 ) depending on whetherXis discrete or continuous. In particular, takingg(x) = cxfor some constantcleads to the formula E(cX) = cE(X). (A.8) Ofparticularimportancearethefunctions g(x) = xn.Forsuchafunction E[g(X)]isknown as thenth momentof X. We define thevarianceof Xby Var(X) = E[X−E(X)]2 = E(X2)−E(X)2. (A.9) 498 A REVIEW OF PROBABILITY THEORY It is clear from (A.8) that for any constantc Var(cX) = c2Var(X). (A.10) The positive square root of Var(X) is known as thestandarddeviation ofX. The smaller the variance, the more likely it is that values ofX are close to the mean. A formal statement along these lines is given byChebyshev’s inequality, which states that for k> 0, P(|X−E(X)|) ≥ k≤ Var(X) k2 . (A.11) Up to this point, we have talked first about events and probabilities, and second about random variables and expectations. It is interesting to note that, if we wish, we can subsume events and probabilities under the latter category, and speak only of random variables and expectations. For any eventAwe define a random variableI A that takes the value 1 on points in Aand the value 0 on points not inA. These are known asindicator random variables.I n thiswaytheevent Acanbeviewedasarandom variableand P(A)isequalto E[IA],ascanbe easily verified. A.6 Expectation in terms of the distribution function Itisoftendesirabletoexpressexpectationsintermsofthedistributionfunctionratherthanthe densityorprobabilityfunction.Inplaceof F,itismoreconvenienttoworkwiththefunction s(t) = 1−F(t) = P(T > t). For a simple example, consider a discrete random variableX: E(X) = f(1)+2f(2)+3f(3)+⋯ = f(1)+f(2)+f(3)+⋯ +f(2)+f(3)+⋯ +f(3)+⋯ Summing by rows, we get E(X) = ∞∑ k=0 s(k). (A.12) In the continuous case we can similarly derive E(X) = ∫ ∞ 0 s(t)dt. (A.13) JOINT DISTRIBUTIONS 499 We will in fact develop a more general formula. Supposegis a differentiable function such thatE[g(X)] exists and is finite. Noting thatf(t) =− s′(t), we integrate by parts to obtain ∫ N 0 g(t)f(t)dt=− g(N)s(N)+g(0)+∫ N 0 g′(t)s(t)dt. In the case where g(N)s(N) → 0, as N → ∞ (A.14) we can conclude that E[g(X)] = g(0)+∫ ∞ 0 g′(t)s(t)dt. (A.15) A similar derivation in the discrete case shows that, for any functiongsatisfying (A.14) for integer values ofN, E[g(X)] = g(0)+ ∞∑ k=0 [g(k+1)−g(k)]s(k). (A.16) Condition(A.14)isautomaticallysatisfiedif gisbounded,since s(N)tendsto0as Ntends to ∞,o ri fXis bounded, since thens(N) = 0 for sufficiently highN. It is also satisfied for monotoneg. Suppose, for example,gis increasing and nonnegative. Then (in the continuous case), g(N)s(N) = g(N)∫ ∞ N f(t)dt≤ ∫ ∞ N g(t)f(t)dt, and the last term must approach 0, by the hypothesis thatE[g(X)] = ∫ ∞ 0 g(t)f(t)dtis finite. A.7 Joint distributions Supposewehavetworandomvariables XandYdefinedonthesamesamplespace.Formany applications, we are interested not only in the distribution of each random variable, but also in thejointdistribution, which gives information on how the two are related. When both are discrete, we can describe this by ajointprobabilityfunction. This is a two-variable function f X,Y defined by fX,Y(k,n) = P(X = kand Y = n). (As in the single-variable case, we sometimes omit the subscripts onf when no confusion results.) WhenX and Y are both continuous we define ajointdensityfunctionf X,Y.T h i si sa 500 A REVIEW OF PROBABILITY THEORY two-variable function, such that for suitable regionsA in the plane, the probability that the point (X,Y) lies inAis given by ∫∫ A fX,Y(x,y)dxdy. (A.17) The probability or density functions associated with the component random variables are obtained from the corresponding joint function as the so-calledmarginaldistributions, namely fX(k) = ∞∑ n=0 fX,Y(k,n)o r fX(x) = ∫ ∞ 0 fX,Y(x,y)dy, (A.18) according as the random variables are both discrete or both continuous. As in the single random variable case, we assume that joint density functions exist unless otherwise indicated, but readers should be cautioned that for two continuous distributions,X, and Y, a joint density function need not exist even when bothfX and fY do. For example, if X = Y, then (X,Y) lies in the regionA= {(x,y): x= y} with probability 1, but the double integral of any two-variable function overAwill equal 0. Forafunction goftwovariablesdefinedonasetthatcontainsallthepoints (X(𝜔),Y(𝜔)) where𝜔 is in the sample spaceΩ, it can be shown, analogously to (A.7), that E[g(X,Y)] = ∞∑ k=0 ∞∑ n=0 g(k,n)fX,Y(k,n) (A.19) in the discrete case, or E[g(X,Y)] = ∫ ∞ 0 ∫ ∞ 0 g(x,y)fX,Y(x,y)dxdy (A.20) in the continuous case. An important example is the functiong(xy) = xy, which is used to define thecovarianceof Xand Y given by Cov(X,Y) = E[(X−E(X))(Y−E(Y))] = E(XY)−E(X)E(Y). (A.21) WhenCov(X,Y) > 0,itmeans thatthetworandom variables move inthesamedirection. High (low) values of one will tend to imply high (low) values of the other. Cov (X,Y) < 0 means the two random variables move in opposite directions. When Cov(X,Y) = 0, we say thatXandY are uncorrelated. For another frequently used example, takeg(x,y) =x+y. Then E(X+Y) = ∫ ∞ 0 ∫ ∞ 0 (x+y)fX,Y(x,y)dxdy= ∫ ∞ 0 ∫ ∞ 0 xfX,Y(x,y)dxdy + ∫ ∞ 0 ∫ ∞ 0 yfX,Y(x,y)dydx. CONDITIONING AND INDEPENDENCE FOR RANDOM VARIABLES 501 (Note the interchange of variables in the second integral.) Now using (A.20) we derive the often used result that E(X+Y) = E(X)+E(Y). (A.22) We need not confine ourselves to just two random variables, and may want to consider functions ofn random variables defined on the same sample space, wheren is any positive integer. In the case of a sum, formula (A.22) extends by induction to E [ n∑ i=1 Xi ] = n∑ i=1 E(Xi). (A.23) Another key formula involving a sum of random variables is Var [ n∑ i=1 Xi ] = n∑ i=1 Var(Xi)+2 ∑ i<j Cov(Xi,Xj). (A.24) A.8 Conditioning and independence for random variables The notions of independence and conditioning can be extended from events to random vari- ables. We say that two random variablesX and Y are independentif their associated events are independent. That is, for any two Borel setsAandB, P(X ∈ AandY ∈ B) = P(X ∈ A)P(Y ∈ B). Asin(A.4),wecanextendthisdefinitiontoanycollectionofrandomvariablesbyrequiring that, given any finite number of random variablesX 1,X2,…,Xn from this collection and any Borel setsA1,A2,…,An,w eh a v e P(X1 ∈ A1,X2 ∈ A2,…,Xn ∈ An) = P(X1 ∈ A1)P(X2 ∈ A2)⋯ P(Xn ∈ An). Equivalent formulations can be made in terms of the joint probability or density, or distribution functions. For example, in the bivariate case, the random variablesX and Y are independent if and only if for all points (x,y), fX,Y(x,y) = fX(x)fY(y), (A.25) or FX,Y(x,y) = FX(x)FY(y). (A.26) This leads immediately to the fact that E(XY) = E(X)E(Y)i f XandYare independent. (A.27) 502 A REVIEW OF PROBABILITY THEORY Aswewouldexpect,twoindependentrandomvariablesareuncorrelated.However, Xand Y can be uncorrelated without being independent (see Section 15.7). If X is a random variable andB an event withP(B) > 0 we can speak of the random variableX|B. This is just the restriction ofXto the sample spaceB. Then E(X|B) = the expectation ofX|B (A.28) with respect to the probability measureP(⋅|B). Conditioning is often a useful tool in computing an expectation, since we can break up the calculation into various cases. Formally we have what is known as thelaw of total expectation, which states that given a partition ofΩ, that is, a collection of pairwise disjoint setsB1,B2,…,Bn with unionΩ, E(X) = n∑ i=1 E(X|Bi)P(Bi). (A.29) TakingXto beIA for an eventAgives thelawoftotalprobability P(A) = n∑ i=1 P(A|Bi)P(Bi). (A.30) A.9 Moment generating functions Given any random variableX,t h emomentgeneratingfunction(m.g.f.) is a functionMX of a real variabletdefined by MX(t) = E[etX], (A.31) provided thatE[etX] exists in some neighbourhood of 0. (There are distributions for which E(etX) =∞ for all positive values oft, in which case the m.g.f. will not exist in any neigh- bourhood of 0.) Recalling the series expansion ex = 1+x+ x2 2! + x3 3! +⋯ , so that etX = 1+tX+ t2X2 2! + t3X3 3! +⋯ , we see that the m.g.f. has the series expansion MX(t) = 1+tE(X)+ t2E(X2) 2! + t3E(X3) 3! +⋯ , PROBABILITY GENERATING FUNCTIONS 503 from which we obtain E(Xn) = the nth derivative ofMX(t) evaluated att= 0. (A.32) Some important facts about m.g.f.’s are as follows Theorem A.1 ForX andY independent MX+Y(t) = MX(t)MY(t). Proof. Using A.26 we have MX+Y(t) = E(et(X+Y)) = E(etXetY) = E(etX)E(etY) = MX(t)MY(t), where we invoke the fact that functions of independent random variables are themselves independent to justify the third equality. The second theorem, which we state without proof, tells us that the m.g.f. determines the distribution when it exists. Theorem A.2 (The uniqueness theorem) Supposethat,fortworandomvariablesX and Y,M X(t)andMY(t)areequalinsomeneighbourhoodofzero.ThenX ∼ Y. This theorem does not say that one can easily compute the distribution function or density function from the m.g.f. The idea is rather that if one recognizes an m.g.f. as being that of a certain known distribution, then the distribution in question must be that distribution. An example of its use will appear in Section A.11.2 below. A.10 Probability generating functions For discrete distributions that take nonnegative integer values, it is often more convenient to use another function closely related to the m.g.f. This is known as theprobabilitygenerating function(p.g.f.) and is defined by P X(t) = ∞∑ k=0 tkfX(k). (A.33) This should not be confused with the probability function as defined above. Given the p.g.f., we can immediately find the complete distribution by f X(k) = P(k) X (0) k! , where the superscript (k) denotes thekth derivative with respect tot. The p.g.f. is related to the m.g.f. because we can write it as PX(t) = E(tX) = E(elogtX) = MX(logt), (A.34) 504 A REVIEW OF PROBABILITY THEORY and it follows that MX(t) = PX(et). (A.35) If we find the p.g.f. of a discrete distribution, (A.35) immediately gives us the m.g.f. as well. The following technique will illustrate a good way to remember the p.g.f.’s of familiar distributions. Suppose we start with any functiongwhose power series expansion about zero hasallnonnegativecoefficients(i.e.,allitsderivativesarenonnegativeat0)andwhoseradius of convergence is greater than 1. Then g(t) = a 0 +a1t+a2t2 +⋯ , wherean ≥ 0 for alln. Since g(1) = a0 +a1 +⋯ , such an expansion immediately gives rise to a probability distribution with probability function f(k) = a k g(1), k= 0,1, …. (A.36) The resulting p.g.f. is immediate since, for any random variableXwith this distribution, PX(t) = 1 g(1) ∞∑ k=0 aktk = g(t) g(1). (A.37) Particular applications of this idea are found in Sections A.11.2 and A.11.3 below. To calculate moments given the p.g.f., we could find the m.g.f. by (A.35) and then use (A.32), but we can also use (A.33) directly. Notice, for example, that P′ X (1) = E(X), (A.38) and similarly P′′ X (1) = E[X(X−1)], from which we can derive Var(X) = E[X(X−1)]+E(X)−E(X)2 = P′′ X (1)+P′ X (1)−P′ X (1)2. (A.39) We close this section by noting that (A.34) and (A.35) easily imply that the analogue of Theorem A.1 holds for the p.g.f. That is, ifZis an independent sum ofXandY, then PZ(t) = PX(t)PY(t). (A.40) SOME STANDARD DISTRIBUTIONS 505 A.11 Some standard distributions We review here the main features of some standard distributions. A.11.1 The binomial distribution Suppose we take m independent trials of a random experiment where the outcomes are classified as either ‘success’ or ‘failure’, and the probability of a success isp. The number of successes is a random variable with probability function p(k) = ( m k ) pk(1−p)m−k, k= 0,1, …,m. This is known as a binomial random variable with parametersmand p, and we will denote it by Bin(m,p). It can be generated, as in (A.36), by the function g(t) = (1−p+pt)m (abinomialexpansion,whichisthesourceofthename).Itfollowfrom(A.37)to(A.39)that PBin(m,p)(t) = (1−p+pt)m, E[Bin(m,p)] = mp,V a r [ B i n (m,p)] = mp(1−p). (A.41) A.11.2 The Poisson distribution The Poisson distribution arises as the limit of Binomials where the mean is held constant. Precisely, for any𝜆>0 the Poisson(𝜆) distribution is the limit of Bin(m,𝜆∕m)a smgoes to ∞. It will have a probability function given by p(k) = lim m→∞ m! k!(m−k)! ( 𝜆 m )k( 1− 𝜆 m )m−k = 𝜆k k! limm→∞ [( 1− 𝜆 m )m( 1− 𝜆 m )−k (m(m−1)(m−2)⋯ (m−k+1) mk )] . The first factor in the square brackets approaches e−𝜆, and the other two each approach 1, so we are left with p(k) = e−𝜆𝜆k k!, k= 0,1,2 …, Thisisgeneratedasin(A.36)fromthefunction g(t) = e−𝜆t sowecandeducefrom(A.37) to (A.39) that PPoisson(𝜆)(t) = e𝜆(t−1), E[Poisson(𝜆)] = 𝜆, Var[Poisson(𝜆)] = 𝜆2 +𝜆−𝜆2 = 𝜆. (A.42) 506 A REVIEW OF PROBABILITY THEORY Them.g.f.canbecalculateddirectlyfromthedefinition,butiteasiertoderiveitfrom(A.35). MPoisson(𝜆)(t) = PPoisson(𝜆)(et) = e𝜆(et−1). (A.43) Here is an example to illustrate the use of m.g.f.’s in deducting distributions. Example A.1 IfXhasaPoisson(𝜆)distributionand Y hasaPoisson( 𝜇),distribution.Find the distribution of an independent sumZ = X+Y. Solution. Directly from (A.43) and Theorem A.1 MZ(t) = MX(t)MY(t) = e(𝜆+𝜇)(et−1) and we conclude thatZhas a Poisson(𝜆+𝜇) distribution. This example illustrates a familiar phenomenon. For many common distributions, the sum of an independent collection is just another distribution of the same type but with a different parameter. In the Poisson case, since the parameter equals the mean, it is clear that the parameter for the sum of the random variables must be the sum of the parameters. A.11.3 The negative binomial and geometric distributions Let us again take repeated trials of an experiment where the probability of a ‘success’ isp. Now, however, instead of taking a fixed number of repetitions, we continue until we get a ‘failure’. We then count the number of successes. This is a random variable with p(k) = p k(1−p), k= 0,1, …. It is known as thegeometricdistribution with parameterp(the name is chosen to reflect the fact the values of the probability function form a geometric progression) and will be denoted by Geom(p). IfN is such a random variable, P(N ≥ m) = (1−p) ∞∑ k=m pk = pm. It follows that, for positive integersrand t, P(N ≥ t+r|N ≥ r) = pt+r pr = pt = P(N ≥ t). (A.44) Thissaysthatatanypoint,theprobabilitythatyouwillcontinueforafurther srepetitions is independent of the number of repetitions you have already taken, sometimes referred to as a memorylessfeature of the distribution. Suppose we repeat trials as above, except that instead of continuing until we get a single failurewecontinueuntilweget rfailures.Theprobabilityfunctionforthenumberofsuccesses SOME STANDARD DISTRIBUTIONS 507 is now given by p(k) = ( r+k−1 k ) pk(1−p)r = r(r+1)⋯ (r+k−1) k! pk(1−p)r, k= 0,1, …. (A.45) Itisclearfromthesecondexpressionthat rcanbeanypositivenumberandnotjustaninteger. Wenolongergettheinterpretationofrepeatedtrialsasgivenbutwestillhaveaperfectlyvalid distribution. It is known as thenegative binomialdistribution with parametersr and p, and we denote it by Negbin(r,p). It is generated as in (A.36) from the functiong(t) = (1−pt) −r, which is the source of the name of the distribution. From (A.37) to (A.39) we have that PNegbin(r, p)(t) = ( 1−p 1−pt )r , E[Negbin(r,p)] = rp 1−p, Var[Negbin( r,p)] = rp (1−p)2. (A.46) Theaboveformulasgivethesamequantitiesforthegeometricdistribution,simplybytaking r= 1. There is another parameterization of the negative binomial that is convenient for certain purposes. In place ofp, we use the quantity𝛼 = p∕(1−p). Then 1∕(1−p) = 1+𝛼.I nt e r m s of this parameter, we can write E[ Negbin(r,p)] = r𝛼, Var[Negbin( r,p)] = r𝛼(1+𝛼). A.11.4 The continuous uniform distribution Suppose we choose a point at random in an interval (a,b). If we decide that any interval of a givenlength hisequallylikelyregardlessofwheretheintervalstarts,thenwehaveauniform distributionU. It has a continuous density function given by fU(t) = t b−a, a< t< b, and direct integration shows that E(U) = b−a 2 ,V a r ( U) = (b−a)2 12 . (A.47) A.11.5 The normal distribution One of the most widely used distributions in probability and statistics is the normal. The standardnormal distribution is that with density function given by f(z) = 1 √ 2𝜋 e−z2∕2, −∞ < x< ∞, 508 A REVIEW OF PROBABILITY THEORY whose graph is the familiar bell-shaped curve. This is one instance where we have a random variable that takes negative values. For a standard normal random variableZ E(Z) = 0, Var(Z ) = 1, as shown by (A.32) and (A.49) below. The complete family of normal random variables consists of those of the form. 𝜇 +𝜎Z where 𝜇 is any constant𝜎> 0 and Z has a standard normal distribution. Such a random variable will have mean𝜇 and variance𝜎 2. The importance of normal distributions comes from the famouscentral limit theorem. Suppose we have an independent sequenceX1,X2,… of random variables, each with the same distribution. Then the averagesSn = (X1,X2,…,Xn)∕n converges in some sense to a normal distribution. (We will not give precise details here.) As shown below in Section A.12 it can be difficult to compute the distribution of a sum of random variables. In many cases oneinvokesthecentrallimittheoremtoapproximatethedistributionofasumbyassumingit is normal, and then only the mean and variance need be known. LetΦdenotethecumulativedistributionfunctionofthestandardnormal.Theintegration to compute this from the density function must be done by some numerical method. It used to be customary to publish tables of various values ofΦ but now it can be done by various computer programs. In Excel ®, the formula= NORMSDIST(t) returns the valueΦ(t). For example=NORMSDIST(0.5)returns0,whichisobvioussincethedistributionissymmetric aboutthemeanof0.Foranotherexample, =NORMSDIST(1.645)returns0.95,meaningthat with probability 0.95 a standard normal takes a value less than or equal to 1.645. In most applications we are interested in the inverse calculation. Given𝛼 we we want to find the𝛼-quantile of the distribution, that is the pointt so that the probability of being less than t is 𝛼.T h i si sΦ−1(𝛼), which in Excel® is denoted by NORMSINV(𝛼). For example, taking𝛼 = 0.95, the formula=NORMSINV(0.95) returns 1.645, as we have verified. For the case of a general normalX = 𝜇 +𝜎(Z)w eh a v e X ≤ x if and only if 𝜇 +𝜎(Z) ≤ x if and only if Z ≤ (x−𝜇)∕𝜎, from which it follows that FX(x) =Φ (x−𝜇 𝜎 ) , fX(x) = 1 𝜎fZ (x−𝜇 𝜎 ) (A.48) where we invoke (A.6) for the second equation. It follows moreover that the𝛼-quantile ofX is given by𝜇 +Φ−1(𝛼)𝜎. To calculate the m.g.f. we start withZ, the standard normal. Then MZ(t) = 1√ 2𝜋∫ ∞ −∞ etxe−x2∕2dx. SOME STANDARD DISTRIBUTIONS 509 Wehandlethisbythefamiliar‘completingthesquare’trick.Theexponentintheintegrand is −1 2[x2 −2tx+t2 −t2] =− 1 2[(x−t)2 −t2]. Sincex is the variable of integration here, the terms involvingtcan be factored out, and we write MZ(t) = et2∕2 [ 1√ 2𝜋 ∫ ∞ −∞ e−(x−t)2∕2dx ] . Now the term inside the bracket is just the integral over the entire range of the density function of the normal distribution with meant and standard deviation 1, as we see from (A.48) so it must equal 1. We conclude that MZ(t) = et2∕2. (A.49) To handle the general case, we will use some formulas involving modifications by con- stants. For any constanta, MX+a(t) = E[et(X+a)] = etaE[etX] = etaMX(t). (A.50) For any constantb, MbX(t) = E[etbX] = MX(bt). (A.51) Thegeneralnormalrandomvariable Xwithmean 𝜇andstandarddeviation 𝜎isdistributed as 𝜇 +𝜎Zthat by combining (A.49)–(A.51), we obtain MX(t) = et𝜇+𝜎2t2∕2. (A.52) A.11.6 The gamma and exponential distributions The gamma distribution is produced from the gamma function, defined by Γ(𝛼) = ∫ ∞ 0 x𝛼−1e−xdx, 𝛼> 0. Wecannotevaluatethisintegralintermsofelementaryfunctions.Wecan,however,getsome information by integrating by parts. For𝛼> 1, Γ(𝛼) =− x 𝛼−1 e−x| | ∞ 0 +(𝛼 −1)∫ ∞ 0 x𝛼−2e−xdx= (𝛼 −1)Γ(𝛼 −1). (A.53) Since Γ(1) = 1, it follows by induction that for any positive integern, Γ(n) = (n−1)!. 510 A REVIEW OF PROBABILITY THEORY Choose any𝛽> 0 and consider a more general integral, ∫ ∞ 0 x𝛼−1 e−𝛽xdx= Γ(𝛼) 𝛽𝛼 , whichfollowsbymakingachangeofvariablefrom xto𝛽x.Fromthis,wedefineacontinuous positive-valued random variable with the density function f(x) = 𝛽𝛼 Γ(𝛼)x𝛼−1e−𝛽x, x> 0. This is known as a Gamma distribution with parameters𝛼 and 𝛽 and will be denoted by Gamma(𝛼,𝛽). For such a distributionX MX(t) = 𝛽𝛼 Γ(𝛼) ∫ ∞ 0 x𝛼−1 e−(𝛽−t)xdx= 𝛽𝛼 Γ(𝛼) Γ(𝛼) (𝛽 −t)𝛼 = ( 𝛽 𝛽 −t )𝛼 . (A.54) Evaluating derivatives at 0 then gives E(X) = 𝛼 𝛽, E(X2) = 𝛼(𝛼 +1) 𝛽2 ,V a r ( X) = 𝛼 𝛽2. (A.55) Thespecialcaseofthegammawhen 𝛼 = 1arisesfrequently.Itisknownasan exponential distributionandwillbedenotedbyExp(𝛽 ).Forsuchadistribution Xwehavefromabovethat fX(t) = 𝛽e−𝛽t, sX(t) = e−𝛽t, MX(t) = 𝛽 𝛽 −t, E(X) = 1 𝛽,V a r ( X) = 1 𝛽2. (A.56) Thisistheuniquecontinuousdistributionwhichhasthesame“memoryless”propertythat we observed in the discrete case for the geometric distribution, namely, for any nonegativet, andr P(X > t+r|X > r) = P(X > t). (A.57) A.11.7 The lognormal distribution Another commonly used positive-valued distribution is thelognormal. A random variableY is said to have a lognormal distribution ifY = eX where Xhas a normal distribution. That is, justasthenamesuggests,thelogarithmof Y isnormal.Wecancalculatethemomentsofthis distribution from (A.52). Namely, ifX ∼ N(𝜇,𝜎2) (the normal distribution with mean𝜇 and variance𝜎2), then E(Y) = E(eX) = MX(1) = e𝜇+𝜎2∕2, (A.58) and E(Y2) = MX(2) = e2𝜇+2𝜎2 , (A.59) CONVOLUTION 511 from which we obtain Var(Y) = E(Y2)−(EY)2 = e2(𝜇+𝜎2)(e𝜎2 −1). A.11.8 The Pareto distribution The probability density function of the Pareto (𝜃,𝛼) distribution is proportional to (x+ 𝜃)−(𝛼+1). Since ∫ ∞ 0 (x+𝜃)−(𝛼+1) dx= 𝜃−𝛼 𝛼 , this density function is given by f(x) = 𝛼𝜃𝛼 (x+𝜃)𝛼+1, For this distribution, s(x) = 𝛼𝜃𝛼 ∫ ∞ x (y+𝜃)−(𝛼+1) dy= ( 𝜃 x+𝜃 )𝛼 and, provided𝛼> 1, E(X) = ∫ ∞ 0 s(t)dt= 𝜃 𝛼 −1. Further integration leads to E(X2) = 2𝜃2 (𝛼 −1)(𝛼 −2), provided 𝛼> 2. In general,E(Xk) becomes infinite fork≥ 𝛼. A.12 Convolution A.12.1 The discrete case Convolution is a tool for finding the distribution of a sum of random variables. Consider a simple example. SupposeX takes the values 2, 5 and 8 with probabilities 0.5, 0.3 and 0.2, respectively, whileY takes the values 3, 6 and 9, with probabilities 0.6, 0.3 and 0.1, respectively, and thatX and Y are independent. LetZ = X+Y. Thiscanhappenifandonlyif X = 2and Y = 3.Bytheindependenceassumption,we multiply to get the probability of both these occurrences, and we see that f Z(5) = 0.5×0.6 = 0.30. 512 A REVIEW OF PROBABILITY THEORY Whatistheprobabilitythat Z = 8?Therearetwomutuallyexclusivewaysthatthiscanoccur, namely,X = 2,Y = 6 andX = 5,Y = 3. We deduce that fZ(8) = fX(2)fY(6)+fX(5)fY(3) = 0.5×0.3+0.3×0.6 = 0.33. For the probability thatZ = 11, we must take three terms: fZ(11) = fX(2)fY(9)+fX(5)fy(6)+fX(8)fY(3) = 0.5×0.1+0.3×0.3+0.2×0.6 = 0.26. Proceeding similarly, we can calculatefZ(14) = 0.09 andfZ(17) = 0.02, completing the dis- tribution. Thegeneralrulecanbewrittenasfollows.Let XandYbeindependentrandomvariables, with nonnegative integers as values, and letZ = X+Y. Then fZ(n) = ∑ fX(k)fY(m), (A.60) where the sum is taken over all pairs (k,m)f o rw h i c hk+m= n. Formula(A.60)isfineforsmallexampleslikethis,butincaseswheretherandomvariables take on many more values (perhaps even an infinite number) we want to write the formula in a more systematic way, which will ensure that we do not miss any combinations. In order for a sum of nonnegative integersk+m to sum to a nonnegative integern, we can havek take any value from 0 tonand thenmmust take the valuen−k. We can then write (A.60) as f Z(n) = n∑ k=0 fX(k)fY(n−k). (A.61) Equation (A.61) is the general so-called convolution formula. Note, however, that for small problems like the one above, which we want to do by hand calculation, it can be inefficient compared to (A.60) since we will be adding up many terms of 0. Suppose we want the probability thatZ ≤ n. We can reason in the same way. We needX to take a valuekfrom 0 tonand nowY must take a value less than or equal ton−k,s ow e have F Z(n) = n∑ k=0 fX(k)FY(n−k). (A.62) It is not hard to verify that we can interchangeFandf also write (A.28) as FZ(n) = n∑ k=0 FX(k)fY(n−k). (A.63) This follows simply by changing the variable of summation fromk to n−k. The reader is warned that wecannotwriteFZ(n) = ∑n k=0 FX(k)FY(n−k). Asanillustrateofconvolution,wewillredoExampleA.1,whichwedidbeforebyp.g.f.’s, now using convolution. CONVOLUTION 513 From (A.61), fZ(n) = n∑ k=0 e−𝜆 e−𝜇 𝜆k𝜇n−k k!(n−k)! = e−(𝜆+𝜇)(𝜆+𝜇)n n! n∑ k=0 ( n k )( 𝜆 𝜆+𝜇 )k( 𝜇 𝜆+𝜇 )n−k . The summation on the right hand side is just a binomial expansion [( 𝜆 𝜆+𝜇 ) + ( 𝜇 𝜆+𝜇 )]n = 1, and we conclude thatZhas a Poisson(𝜆+𝜇) distribution. Supposewewantthesumofmorethantwodiscreterandomvariables.Forexample,given independent random variablesX,Y,W, find the distribution ofV = X+Y+W. For cases where there are few nonzero values, we can proceed just as in (A.60) and write fV(n) = ∑ fX(k)fY(m)fW(p), where the sum is taken over all ordered triples (k,m,p) such thatk+m+p= n.F o rt h e general case, however, it may not be so easy to pick out all such triples, and we have to proceed more systematically. The basic procedure is to iterate the calculation. We first apply (A.60) or (A.61) to find the distribution ofZ = X+Y and then apply it again to find the distribution ofV = Z+W. For the general case of a sum ofnindependent random variables, we just iterate thisn−1 times. Example A.2 Let Z = ∑ 5 i=1 Xi, where theXi’s are independent random variables, each takingthevalue0withprobability0.7,1withprobability0.2and2withprobability0.1.Find the probability thatZ = 2. Solution. We can derive this single number by inspection, without having to do the four iterations. One way for the five values to add up to 2 is that one of them is 2 and the rest are zero. Since there are five possibilities for the 2, the probability of this is 5×0.7 4 ×0.1. The only other way is for three of the values to be zero and the other two to be 1. The probability of this 10×0.7 3 ×0.22.S o P(Z = 2) = 5×0.74 ×0.1+10×0.73 ×0.22 = 0.2573. A.12.2 The continuous case We now suppose thatX and Y are independent,continuous, nonnegative random variables, and we want the distribution ofZ = X+Y.I fXtakes the valuexandY takes the valuey,t h e sum will be less than or equal tosif and only if the point (x,y) lies in the region bounded by 514 A REVIEW OF PROBABILITY THEORY thexandyaxes, and the liney= s−x. Integrating the joint density function over this region gives FZ(s) = ∫ s 0 ∫ s−x 0 fX,Y(x,y)dydx. (A.64) Equation(A.64)isinfacttruewithouttheindependenceassumption,butwiththeassumption of independence the integral simplifies considerably. The joint density function factors as f X(x)fY(y), the inner integral becomes ∫ s−x 0 fY(y)dy= FY(s−x) and we can write FZ(s) = ∫ s 0 fX(x)FY(s−x)dx, (A.65) a direct analogue of (A.62). Using Leibniz’s rule for differentiating integrals and the fact thatFY(0) = 0, we have fZ(s) = ∫ s 0 fX(x)fY(s−x)dx, (A.66) a direct analogue of (A.61). The following example serves to indicate that calculating convolutions for continuous distributions can be a very involved procedure, even in the simplest cases. Example A.3 Find the distribution of an independent sumZ = X+Y, where X has a uniform distribution on the interval [0, 2] andY has a uniform distribution on the interval [0, 3]. Solution. Whether we use (A.65) or (A.66) depends on the particular example. In this case, it is easier to use (A.66). See Figure A.1, and notice that it depicts a region of the (x,s) plane (rather than the (x,y) place. It is that portion of g the positive quadrant in the (x,s) plane givenby0 ≤ s≤ 5,0 ≤ x≤ s.Adistributionthatisuniformonaninterval[ a,b]hasadensity function that is a constant (b−a) −1 on this interval and zero elsewhere. Consequently, the integrandin(A.66)takesthevalue1/6when0 ≤ x≤ 2and0 ≤ s−x≤ 3,asindicatedbythe union of regionsR1,R2,R3, and it take the value of 0 elsewhere, as indicated by the union of regionsS1,S2,S3. Now we consider in turn all possible values ofs and the value of the integral in (A.66). CONVOLUTION 515 S1 R1 R2 R3 S2S3 x s s = 2 s = 3 s = 5 x = 2 x = 3 x = 50 Figure A.1 The region of integration in Example A.3 If0 ≤ s≤ 2,thevalueof xinR1 willvaryfrom0to s,sothattheintegrandwillbe1/6for allx. Therefore, fZ(s) = s 6,0 < s≤ 2. If 2≤ s≤ 3, the value ofxin R2 will vary from 0 to 2, so that the integrand will be 1/6 whenx≤ 2 and zero when 2< x≤ s. Therefore, fZ(s) = 1 3,2 ≤ s≤ 3. If 3≤ s≤ 5, then the value ofXinR3 will vary froms−3 to 2, so that the integrand will be 1/6 whens−3 ≤ x≤ 2, but zero when 0≤ x< s−3 or when 2≤ x. Therefore, fZ(s) = 5−s 6 ,3 ≤ s< 5. Of course, sinceZvaries from 0 to 5, we know that fZ(s) = 0, s< 0o rs> 5. A.12.3 Notation and remarks Suppose thatX has probability (or density) functionf, and distribution functionF, whileY has probability (or density) functiongand distribution functionG. The probability (or density) function given in (A.61) (or (A.66)) is denoted byf ∗ g. The distribution function given in (A.62) (or (A.65)) is denoted byF ∗ G.O n em u s tt a k e care with this latter notation as it could induce the error that the reader was warned about 516 A REVIEW OF PROBABILITY THEORY above, of using the c.d.f. in both places when calculating the c.d.f. of the sum. Similarly, given a sum ofnindependent random variables with the same distribution, we usef ∗n for the probability (density) function of the sum andF∗n for the distribution function of the sum. To illustrate, in Example A.2 we would denoted the desired answer asf∗5(2), wheref was the given probability function. As a final remark in this section, we note that all of the above formulas hold for random variablesthattakenegativevalues,withtheexceptionthatwemustreplacethelowerlimiton the integrals and sums by−∞. A.13 Mixtures Suppose you are faced with a choice of two games to play. Game 1 has a return of either 2o r−2 each with probability 1/2, while game 2 has a return of 4 with probability 1/4, 2 with probability 1/4 or−3 with probability 1/2. Racked with indecision, you flip a coin, intendingtopaygame1ifaheadturnsup,orgame2ifatailscomes up.Whatyouarereally playing is amixtureof game 1 and game 2 with equal weights. The resulting return is easily calculatedtobe4withprobability1/8,2withprobability3/8, −2withprobability2/8and −3 with probability 2/8. More generally, we could havenrandom variablesX 1,X2,…,Xn and a probability distribution on {1,2,…,n}, called the mixing distribution. The resulting mixture isarandomvariable Xthattakesavaluefrom Xiwithprobability p(i),where pistheprobability functionofthemixingdistribution.(Ourinitialexample had p(1) = p(2) = 1∕2.)Calculating quantities with respect to the mixed distribution presents no problems as everything follows the same convex combination. We have, for example, f X(s) = n∑ k=1 p(k)fXk(s), FX(s) = n∑ k=1 p(k)FXk(s), MX(s) = n∑ k=1 p(k)MXk(s). (A.67) Letusconsideraslightlytrickiersituation,inwhichwehavecontinuousmixing.Suppose now that we have a whole family of random variablesXt, either all discrete or all continuous, indexed on all the nonnegative reals [0,∞) instead of just the integers. For a mixing distribu- tion,wetakeacontinuousnonnegativerandomvariablewithdensityfunction p.Theresulting mixture is a random variableXwith density function fX(s) = ∫ ∞ 0 p(t)fXt(s)dt. (A.68) References Arrow, K.J. (1963). Uncertainty and the welfare of medical care.American Economic Review53, 941–973. Artzner, P., Delbaen, F., Eber, J.M. and Heath, D. (1999). Coherent measures of risk.Math Finance9, 203–228. Bj¨ork, T. (2009).Arbitrage Theory in Continuous Time, 3rd edn. New York, NY: Oxford University Press. Bowers,N.,Gerber,H.,Hickman,J.,Jones,D.andNesbitt,C.(1997). Actuarial Mathematics,2ndedn. Schaumburg, IL: Society of Actuaries. Brillinger, D.R. (1961). A justification of some common laws of mortality.Transactions of the Society of ActuariesXIII, 116–119. Broverman, S. (2010).Mathematics of Investment and Credit, 5th edn. Winstead CT: ACTEX. B¨ulhman,H.andGisler,A.(2005). A Course in Credibility Theory.Berlin-Heidelberg:Springer-Verlag. Carri`ere, J. (1994a). An investigation of the Gompertz law of mortality.Actuarial Research Clearing House,2 . Carri`ere,J.(1994b).Dependentdecrementtheory. Transactions of the Society of ActuariesXLVI,45–65. Daniel,J.W.andVaaler,L.J.F.(2009). Mathematical Interest Theory,2ndedn.MathematicalAssociation of America. Dickson, D., Hardy, M. and Waters, H. (2013).Actuarial Mathematics for Life Contingent Risks, Cambridge: Cambridge University Press. Etheridge, A. (2002).A Course in Financial Calculus, Cambridge: Cambridge University Press. Frees, E., Carri`ere, J. and Valdez, E. (1996). Annuity valuation with dependent mortality.Journal of Risk and Insurance63, 229–261. Frees, E. and Valdez E. (1998). Understanding relationships using copulas.North American Actuarial Journal 2, 1–25. Gerber,H.andShiu,E.S.(1998).Onthetimevalueofruin. North American Actuarial Journal2,48–78. Grandel, J. (1991).Aspects of Risk Theory. New York, NY: Springer-Verlag. Herzog, T.N. (1999).Introduction to Credibility Theory, 3rd edn. Winstead, CT: ACTEX. Hoel,P.,Port,S.andStone,C.(1972). Introduction to Stochastic Processes,Boston:HoughtonMifflin. Fundamentals of Actuarial Mathematics, Third Edition. S. David Promislow. © 2015 John Wiley & Sons, Ltd. Published 2015 by John Wiley & Sons, Ltd. Companion website: http://www.wiley.com/go/actuarial 518 REFERENCES Hull,J.C.(2014). Options, Futures, and other Derivative Securities,9thedn,Cambridge,UpperSaddle River, NJ: Prentice Hall. Jones, B.L. (1997). Stochastic models for continuing care retirement communities.North American Actuarial Journal1, 50–73. Jordan, C.W. (1967).Life Contingencies, 2nd edn. Schaumburg, IL: Society of Actuaries. Kaas, R., Goovaerts, Dhaene, M.J. and Denuit, M. (2008).Modern Actuarial Risk Theory, 2nd edn. Springer-Verlag. Kemeny, J.G. and Snell, J.L. (1963).Finite Markov Chains. Princeton, NJ: D. Van Nostrand. Klugman, S., Panjer, H. and Willmot, G. (2012).Loss Models, 4th edn. Hoboken, NJ: John Wiley & Sons. London, D. (1997).Survival Models and their Estimation, 3rd edn. Winstead CT: ACTEX. McDonald, R.L. (2012).Derivatives Markets, 3rd edn. Upper Saddle River, NJ: Prentice Hall. Mikosch, T. (1998).Elementary Stochastic Calculus-with Finance in View,R i v e rE d g e ,N J :W o r l d Scientific. Nelsen, R.B. (1999).An Introduction to Copulas, Lecture Notes in Mathematics 139. New York, NY: Springer-Verlag. Norberg, R. (2008). Multi-state models for life insurance mathematics. In: E. Melnick and B. Everitt (eds.), Encyclopedia of Quantitative Risk Analysis and Assessment. Chichester: Wiley-Blackwell. Powers, M.R. (1995). A theory of risk, return, and solvency.Insurance: Mathematics and Economics 17 , 101–118. Promislow,S.D.(1980).Anewapproachtothetheoryofinterest. Transactions of the Society of Actuaries XXXII, 53–92. Promislow, S.D. (1991). Select and ultimate models in multiple decrement theory.Transactions of the Society of ActuariesXLIII, 281–300. Promislow, S.D. (1997). Classification of usurious loans. In: M. Sherris (ed.),Proceedings of the 7th International AFIR Colloquium. Institute of Actuaries of Australia. Ross, S.M. (2010).Introduction to Probability Models, 10th edn. San Diego, CA: Academic Press. Rothschild, M. and Stiglitz, J. (1970). Increasing risk: A definition.Journal of Economic Theory2, 225–243. Royden, H.L.(1988).Real Analysis, 3rd edn. New York, NY: Macmillan. Rudin, W.R. (1976).Principles of Mathematical Analysis, 3rd edn. New York, NY: McGraw-Hill. Shaked, M. and Shantikumar, J. G. (2007).Stochastic Orders. Springer. Steland, A. (2012).Financial Statistics and Mathematical Finance, Chichester: John Wiley & Sons. Teichroew, D., Robichek, A.A. and Montalbano, M. (1965a). Mathematical analysis of rates of return under certainty.Management Science11, 395–403. Teichroew, D., Robichek, A.A. and Montalbano, M. (1965b). An analysis of criteria for investment and financing decisions under certainty.Management Science12, 151–179. Tennenbein, A. and Vanderhoof, I. (1980). New mathematical laws of select and ultimate mortality. Transactions of the Society of ActuariesXLIV, 509–538. Young,V.R.(2004).Premiumprinciples.In:J.F.TeugelsandB.Sundt(eds.), Encyclopedia of Actuarial Science. Chichester: John Wiley & Sons. Notation index The following is a list of the major symbols which are used in the book. For the most part, the first page they appear on is listed. An exception is the notation in Appendix A, where the first appearance in that chapter is noted. This list excludes that part of the standard actuarial notation which is not used in the main body of the text. The latter can be found in the appropriatesectionsofChapters2–6,8and10entitled‘StandardNotationandTerminology’. Chapter 2 ̈a(c;v)1 5 a ∗ b B k(c;v)2 1 ̃Bk(c)2 5 kc 21 kc 21 c ◦ k 25 dk 12 ek 17 ik 12 kV(c;v)2 1 v(k)1 1 v(k,n)1 0 v ◦ k 25 Valn(c;v)2 6 Δb 17 ∇b 18 Chapter 3 d x 39 ex 42 e◦ x 42 npx 40 nqx 40 px 40 qx 40 𝓁x 39 𝜔 40 Chapter 4 ̈ax(c)4 9 (1∞)5 1 yx(k)5 0 ̈a[x]+k(c)5 2 Chapter 5 A x(b)6 2 wx(k)6 9 ̈A[x]+k(c)7 0 Chapter 6 𝜂 k 81 kV 78 Fundamentals of Actuarial Mathematics, Third Edition. S. David Promislow. © 2015 John Wiley & Sons, Ltd. Published 2015 by John Wiley & Sons, Ltd. Companion website: http://www.wiley.com/go/actuarial 520 NOTATION INDEX Chapter 7 ̈a(m)(c;y)9 9 a(m)(c;y) 104 ̈a(m) x (c) 101 d(m) 100 i(m) 100 𝛼(m) 102 𝛽(m) 102 Chapter 8 ̄a(c;y) 113 ̄ax(c) 115 ̄Ax(b) 119 𝛼(∞) 116 𝛽(∞) 116 𝛿y(t) 113 𝛿 114 𝜆x(t) 119 𝜇(x) 118 𝜇x(t) 118 Chapter 9 𝓁 [x]+k 139 sp[x]+t 138 sq[x]+t 137 q[x]+t 138 Chapter 10 ̈axy(c) 145 ̄axy(c) 145 ̈axy(c) 147 Axy(c) 148 ̄Axy(c) Axy(c) ̄Axy(c) 148 A1 xy (b) 152 A2 xy (b) 152 ̄A1 xy (b) 153 ̄A2 xy (b) 153 npxy 143 npxy 146 nqxy 146 nqxy 153 sq1 xy 147 𝜇xy(t) 147 Chapter 11 ̄A(j) x (b) 166 d(j) x 163 d(𝜏) x 163 𝓁(𝜏) x 163 𝓁(j) x 164 np(j) x 165 np(𝜏) x 160 nq(j) x 164 q(j) x 164 q′(j) x 171 𝜇(j) x (t) 165 Chapter 12 AS k 187 Prk 190 Πk 194 Chapter 13 AV k 201 COIk 202 Sx 205 Chapter 14 f x(t) 220 sx(t) 220 T◦u 216 ̃T 217 Chapter 15 ̈ãT (c;v) 229 ̄aT (c,v) 231 ÃT (b,v) 225 ̄AT (b;v) 226 CV(X) 238 rL 234 L 235 Chapter 16 ̂FX 252 ̂fX 252 Chapter 17 f T1,T2,…,TM (t1,t2,…,tm) 260 FT1,T2,…,TM (t1,t2,…,tm) 260 NOTATION INDEX 521 sT1,T2,…,TM (t1,t2,…,tm) 260 FT,J(t,j) 262 fT,J(t,j) 264 sT,J(t,j) 264 𝜇T,J(t,j) 266 Chapter 18 p xy(k,n) 284 pxy 284 o(h) 294 ∼ 284 Chapter 19 p ii(s,t) 311 pii(s,t) 314 𝜇ij(t) 312 Chapter 20 E k(W) 350 EΠ(W) 366 Chapter 21 ⟨N,X⟩ 379 S + 384 (X −d)+ 388 X ∧d 388 Chapter 22 VaR 413 TVaR 𝛼 413 Chapter 23 D(u) 423 J(u) 433  440 𝜓(u) 420 𝜓 k(u) 436 𝜓(u,t) 420 Chapter 24 E(X|Y) 453 Var(X|Y) 455 ∝ 459 Appendix A P(A) 477 P(A|B) 479 f X(x) 480 FX(x) 480 E(X) 481 Var(X) 481 s(t) 482 Cov(X,Y) 484 E(X|B) 486 f ∗ g 499 F ∗ G 499 f ∗n 499 F∗n 499 MX(t) 486 PX(t) 487