Questions and Answers of Essentials Of Stochastic Processes

(General birth and death chains). The state space is f0; 1; 2; : : :g and the transition probability has p.x; x C 1/ D px p.x; x [1] 1/ D qx forx > 0 p.x; x/ D rx for x 0 while the other p.x; y/ D 0.
(Algorithmic efficiency). The simplex method minimizes linear functions by moving between extreme points of a polyhedral region so that each transition decreases the objective function. Suppose there
(Coupon collector’s problem). We are interested now in the time it takes to collect a set of N baseball cards. Let Tk be the number of cards we have to buy before we have k that are distinct.
(Sucker bet). Consider the following gambling game. Player 1 picks a three coin pattern (for example HTH) and player 2 picks another (say THH). A coin is flipped repeatedly and outcomes are recorded
(Ehrenfest chain). Consider the Ehrenfest chain, Example 1.2, with transition probability p.i; i C 1/ D .N [1] i/=N, and p.i; i [1] 1/ D i=N for 0 i N. Let n D ExXn.(a) Show that nC1 D 1 C .1 [1]
(Wright–Fisher model). Consider the chain described in Example 1.7.p.x; y/ D N y !.[1]x /y.1 [1] [1]x/N[1]y where [1]x D .1 [1] u/x=N C v.N [1] x/=N.(a) Show that if u; v > 0, then limn!1 pn.x; y/
(Bishop’s random walk). A bishop can move any number of squares diagonally.Let Xn be the sequence of squares that results if we pick one of bishop’s legal moves at random. Find(a) the stationary
(King’s random walk). A king can move one squares horizontally, vertically, or diagonally. Let Xn be the sequence of squares that results if we pick one of king’s legal moves at random. Find(a)
(Random walk on a clock). Consider the numbers 1; 2; : : : 12 written around a ring as they usually are on a clock. Consider a Markov chain that at any point jumps with equal probability to the two
(Library chain). On each request the i th of n possible books is the one chosen with probability pi . To make it quicker to find the book the next time, the librarian moves the book to the left end
(Bernoulli–Laplace model of diffusion). Consider two urns each of which contains m balls; b of these 2m balls are black, and the remaining 2m[1]b are white.We say that the system is in state i if
(Queue with impatient customers). Customers arrive at a single server at rate[1] and require an exponential amount of service with rate. Customers waiting in line are impatient and if they are not in
(Kolmogorov cycle condition). Consider an irreducible Markov chain with state space S. We say that the cycle condition is satisfied if given a cycle of states x0; x1; : : : ; xn D x0 with q.xi[1]1;
(Detailed balance for three state chains). Consider a chain with state space f1; 2; 3g in which q.i; j / > 0 if i ¤ j and suppose that there is a stationary distribution that satisfies the detailed
(Two queues in series). Consider a two station queueing network in which arrivals only occur at the first server and do so at rate 2. If a customer finds server 1 free he enters the system; otherwise
(When did the chicken cross the road?). Suppose that traffic on a road follows a Poisson process with rate [1] cars per minute. A chicken needs a gap of length at least c minutes in the traffic to
(Queues in series). Consider a k station queueing network in which arrivals to server i occur at rate [1]i and service at station i occurs at rate i . In this problem we examine the special case of
(Feed-forward queues). Consider a k station queueing network in which arrivals to server i occur at rate [1]i and service at station i occurs at rate i. We say that the queueing network is
In the putback option at time 3 you can buy the stock for the lowest price seen in the past and the sell it at its current price for a profit of V3 D S3 min 0m3 Sm Suppose that the stock follows
Suppose Microsoft stock sells for 100 while Netscape sells for 50. Three possible outcomes of a court case will have the following impact on the two stocks.Microsoft Netscape 1 (win) 120 30 2 (draw)
The Cornell hockey team is playing a game against Harvard that it will either win, lose, or draw. A gambler offers you the following three payoffs, each for a $1 bet win lose draw Bet 1 0 1 1.5 Bet 2
It was crucial for our no arbitrage computations that there were only two possible values of the stock. Suppose that a stock is now at 100, but in 1 month may be at 130, 110 or 80 in outcomes that we
GI=G=1 queue. Let 1; 2; : : : be independent with distribution F and Let 1; 2; : : : be independent with distribution G. Define a Markov chain by XnC1 D .Xn C n nC1/C where yC D maxfy; 0g. Here Xn
Lyapunov functions. Let Xn be an irreducible Markov chain with state space f0; 1; 2; : : :g and let 0 be a function with limx!1 .x/D1, and Ex.X1/ .x/ when x K. Then Xn is recurrent. This
Expectations of hitting times. Consider a Markov chain state space S. Let A S and suppose that C D S A is a finite set. Let VA D minfn 0 W Xn 2 Ag be the time of the first visit to A. Suppose
Hitting probabilities. Consider a Markov chain with finite state space S. Let a and b be two points in S, let D Va ^ Vb, and let C D S fa; bg. Suppose h.a/ D 1; h.b/ D 0, and for x 2 C we have
Let Zn be a branching process with offspring distribution pk with p0 > 0and D Pk kpk > 1. Let ./ D P1 kD0 pkk. (a) Show that E.ZnC1 jZn/ D ./Zn .(b) Let be the solution
A branching process can be turned into a random walk if we only allow one individual to die and be replaced by its offspring on each step. If the offspring distributions is pk and the generating
Consider a favorable game in which the payoffs are 1, 1, or 2 with probability 1/3 each. Use the results of Example 5.12 to compute the probability we ever go broke (i.e., our winnings Wn reach $0)
Generating function of the time of gambler’s ruin. Continue with the set-up of the previous problem. (a) Use the exponential martingale and our stopping Theorem to conclude that if 0, then ex
Variance of the time of gambler’s ruin. Let 1; 2; : : : be independent with P. i D 1/ D p and P. i D 1/ D q D 1 p where p < 1=2. Let Sn D S0 C 1 C C n. In Example 4.3 we showed that if V0 D
Mean time to gambler’s ruin. Let Sn D S0CX1C CXn where X1;X2; : : :are independent with P.Xi D 1/ D p < 1=2 and P.Xi D 1/ D 1 p. Let V0 D minfn 0 W Sn D 0g. Use Wald’s equation to
Let Sn D X1 C C Xn where the Xi are independent with EXi D 0 and var .Xi / D 2. (a) Show that S2 n n2 is a martingale. (b) Let D min fn W jSnj > ag. Use Theorem 5.13 to show that E
General birth and death chains. The state space is f0; 1; 2; : : :g and the transition probability has p.x; x C 1/ D px p.x; x 1/ D qx forx > 0 p.x; x/ D 1 px qx for x 0 while the other p.x;
An unfair fair game. Define random variables recursively by Y0 D 1 and for n 1; Yn is chosen uniformly on .0; Yn1/. If we let U1; U2; : : : be uniform on.0; 1/, then we can write this sequence as
Suppose that in Polya’s urn there are r red balls and g green balls at time 0.show that X D limn!1 Xn has a beta distribution.g C r 1/Š.g 1/Š.r 1/Šxg1.1 x/r1
Suppose that in Polya’s urn there is one ball of each color at time 0. Let Xn be the fraction of red balls at time n. Use Theorem 5.13 to conclude that P.Xn 0:9 for some n/ 5=9.
Lognormal stock prices. Consider the special case of Example 5.5 in which Xi D ei where i D normal.; 2/. For what values of and is Mn D M0 X1 Xn a martingale?
Let Xn be the Wright–Fisher model with no mutation defined in Example 1.9.(a) Show that Xn is a martingale and use Theorem 5.14 to conclude that Px.VN
Four children are playing two video games. The first game, which takes an average of 4min to play, is not very exciting, so when a child completes a turn on it they always stand in line to play the
At a local grocery store there are queues for service at the fish counter (1), meat counter (2), and caf´e (3). For i D 1; 2; 3 customers arrive from outside the system to station i at rate i , and
At present the Economics department and the Sociology department each have one typist who can type 25 letters a day. Economics requires an average of 20 letters per day, while Sociology requires only
Customers arrive at a two-server station according to a Poisson process with rate . Upon arriving they join a single queue to wait for the next available server.Suppose that the service times of the
Customers arrive at a carnival ride at rate . The ride takes an exponential amount of time with rate , but when it is in use, the ride is subject to breakdowns at rate ˛. When a breakdown occurs
Customers arrive at the Shortstop convenience store at a rate of 20 per hour.When two or fewer customers are present in the checkout line, a single clerk works and the service time is 3min. However,
Consider a taxi station at an airport where taxis and (groups of) customers arrive at times of Poisson processes with rates 2 and 3 perminute. Suppose that a taxi will wait no matter how many other
Excited by the recent warm weather Jill and Kelly are doing spring cleaning at their apartment. Jill takes an exponentially distributed amount of time with mean 30 min to clean the kitchen. Kelly
A submarine has three navigational devices but can remain at sea if at least two are working. Suppose that the failure times are exponential with means 1, 1.5, and 3 years. Formulate a Markov chain
A small company maintains a fleet of four cars to be driven by its workers on business trips. Requests to use cars are a Poisson process with rate 1.5 per day.A car is used for an exponentially
Brad’s relationship with his girl friend Angelina changes between Amorous, Bickering, Confusion, and Depression according to the following transition rates when t is the time in months.A B C D A 4
We now take a different approach to analyzing the Duke Basketball chain, Example 4.11.0 1 2 3 0 3 2 1 0 1 0 5 5 0 2 1 0 2:5 1.5 3 6 0 0 6(a) Find g.i / D Ei.V1/ for i D 0; 2; 3. (b) Use the
Consider the two queues in series in Problem 4.8. (a) Use the methods of Sect. 4.4 to compute the expected duration of a busy period. (b) calculate this from the stationary distribution.
A taxi company has three cabs. Calls come in to the dispatcher at times of a Poisson process with rate 2 per hour. Suppose that each requires an exponential amount of time with mean 20 min, and that
There are two tennis courts. Pairs of players arrive at rate 3 per hour and play for an exponentially distributed amount of time with mean 1 h. If there are already two pairs of players waiting new
Consider a barbershop with one barber who can cut hair at rate 4 and three waiting chairs. Customers arrive at a rate of 5 per hour. (a) Argue that this new setup will result in fewer lost customers
Consider a barbershop with two barbers and two waiting chairs. Customers arrive at a rate of 5 per hour. Customers arriving to a fully occupied shop leave without being served. Find the stationary
Customers arrive at a full-service one-pump gas station at rate of 20 cars per hour. However, customers will go to another station if there are at least two cars in the station, i.e., one being
A computer lab has three laser printers and five toner cartridges. Each machine requires one toner cartridges which lasts for an exponentially distributed amount of time with mean 6 days. When a
A computer lab has three laser printers that are hooked to the network.A working printer will function for an exponential amount of time with mean 20 days. Upon failure it is immediately sent to the
A computer lab has three laser printers, two that are hooked to the network and one that is used as a spare. A working printer will function for an exponential amount of time with mean 20 days. Upon
There are 15 lily pads and six frogs. Each frog at rate 1 gets the urge to jump and when it does, it moves to one of the nine vacant pads chosen at random. Find the stationary distribution for the
Three frogs are playing near a pond. When they are in the sun they get too hot and jump in the lake at rate 1. When they are in the lake they get too cold and jump onto the land at rate 2. Let Xt be
Solve the previous problem in the concrete case 1 D 1=24; 2 D 1=30; 3 D 1=84;1 D 1=3;2 D 1=5, and 3 D 1=7.
A machine is subject to failures of types i D 1; 2; 3 at rates i and a failure of type i takes an exponential amount of time with rate i to repair. Formulate a Markov chain model with state space
A hemoglobin molecule can carry one oxygen or one carbon monoxide molecule. Suppose that the two types of gases arrive at rates 1 and 2 and attach for an exponential amount of time with rates 3 and
Two people who prepare tax forms are working in a store at a local mall. Each has a chair next to his desk where customers can sit and be served. In addition there is one chair where customers can
(a) Consider the special case of the previous problem in which 1 D 2 D 1, and 1 D 2 D 3, and find the stationary probabilities. (b) Suppose they upgrade their telephone system so that a call to
Two people are working in a small office selling shares in a mutual fund.Each is either on the phone or not. Suppose that salesman i is on the phone for an exponential amount of time with rate i and
Consider the set-up of the previous problem but now suppose machine 1 is much more important than 2, so the repairman will always service 1 if it is broken.(a) Formulate aMarkov chain model for the
Consider two machines that are maintained by a single repairman. Machine i functions for an exponentially distributed amount of time with rate i before it fails.The repair times for each unit are
A small computer store has room to display up to three computers for sale.Customers come at times of a Poisson process with rate 2 per week to buy a computer and will buy one if at least one is
A salesman flies around between Atlanta, Boston, and Chicago as follows.A B C A 4 2 2 B 3 4 1 C 5 0 5(a) Find the limiting fraction of time she spends in each city. (b) What is her average number
Suppose that the limiting age distribution in (3.9) is the same as the original distribution. Conclude that F.x/ D 1 ex for some > 0.
While visiting Haifa, Sid Resnick discovered that people who wish to travel from the port area up the mountain frequently take a shared taxi known as a sherut.The capacity of each car is five people.
Each time the frozen yogurtmachine at the mall breaks down, it is replaced by a new one of the same type. (a) What is the limiting age distribution for the machine in use if the lifetime of a machine
The city of Ithaca, New York, allows for 2-h parking in all downtown spaces.Methodical parking officials patrol the downtown area, passing the same point every 2 h. When an official encounters a car,
Show that chain in Exercise 1.38 with transition probability is 1 2 3 4 1 1=2 1=2 0 0 2 2=3 0 1=3 0 3 3=4 0 0 1=4 4 1 0 0 0 is a special case of the age chain. Use this observation and the previous
Consider the discrete renewal process with fj D P.t1 D j/ and Fi D P.t1 > i/. (a) Show that the age chain has transition probability q.j; j C 1/ D FjC1 Fj q.j; 0/ D 1 FjC1 Fj D fjC1 Fj for j 0(b)
A scientist has a machine for measuring ozone in the atmosphere that is located in the mountains just north of Los Angeles. At times of a Poisson process with rate 1, storms or animals disturb the
People arrive at a college admissions office at rate 1 per minute. When k people have arrive a tour starts. Student tour guides are paid $20 for each tour they conduct. The college estimates that it
A machine tool wears over time and may fail. The failure time measured in months has density fT .t / D 2t=900 for 0 t 30 and 0 otherwise. If the tool fails it must be replaced immediately at a
Consider the set-up of Example 3.4 but now suppose that the car’s lifetime h.t/ D et . Show that for any A and B the optimal time T D 1. Can you give a simple verbal explanation?
Random Investment. An investor has $100,000. If the current interest rate is i% (compounded continuously so that the grow per year is exp.i=100/), he invests his money in a i year CD, takes the
In the Duke versusMiami football game, possessions alternate between Duke who has the ball for an average of 2 min and Miami who has the ball for an average of 6min. (a) In the long run what fraction
A worker has a number of machines to repair. Each time a repair is completed a new one is begun. Each repair independently takes an exponential amount of time with rate to complete. However,
A young doctor is working at night in an emergency room. Emergencies come in at times of a Poisson process with rate 0.5 per hour. The doctor can only get to sleep when it has been 36 min (0.6 h)
One of the difficulties about probability is realizing when two different looking problems are the same, in this case dealing cocaine and fighting fires. In Problem 2.26, calls to a fire station
A cocaine dealer is standing on a street corner. Customers arrive at times of a Poisson process with rate . The customer and the dealer then disappear from the street for an amount of time with
Counter processes. As in Example 1.5, we suppose that arrivals at a counter come at times of a Poisson process with rate . An arriving particle that finds the counter free gets registered and then
A policeman cruises (on average) approximately 10 min before stopping a car for speeding. Ninety percent of the cars stopped are given speeding tickets with an$80 fine. It takes the policeman an
Three children take turns shooting a ball at a basket. They each shoot until they misses and then it is next child’s turn. Suppose that child i makes a basket with probability pi and that
In front of terminal C at the Chicago airport is an area where hotel shuttle vans park. Customers arrive at times of a Poisson process with rate 10 per hour looking for transportation to the Hilton
The times between the arrivals of customers at a taxi stand are independent and have a distribution F with mean F . Assume an unlimited supply of cabs, such as might occur at an airport. Suppose
Thousands of people are going to a Grateful dead concert in Pauley Pavillion at UCLA. They park their the foot cars on several of the long streets near the arena.There are no lines to tell the
The weather in a certain locale consists of alternating wet and dry spells.Suppose that the number of days in each rainy spell is a Poisson distribution with mean 2, and that a dry spell follows a
Suppose that the number of calls per hour to an answering service follows a Poisson process with rate 4. Suppose that 3/4’s of the calls are made by men, 1/4 by women, and the sex of the caller is
Customers arrive at a bank according to a Poisson process with rate 10 per hour. Given that two customers arrived in the first 5min, what is the probability that(a) both arrived in the first 2min.
Policy holders of an insurance company have accidents at times of a Poisson processwith rate . The distribution of the time R until a claim is reported is random with P.R r/ D G.r/ and ER D . (a)
Ignoring the fact that the bar exam is only given twice a year, let us suppose that new lawyers arrive in Los Angeles according to a Poisson process with mean 300 per year. Suppose that each lawyer
Calls originate from Dryden according to a rate 12 Poisson process.Three-fourth are local and one-fourth are long distance. Local calls last an average of 10 min, while long distance calls last an

Showing 1 - 100 of 569