In the situation of Exercise 3, suppose that there are two sites, and the particles sitting in these sites exhibit independent behavior. Let X_t be the total number of particles in both sites.
Show that X_t is a Markov chain and write its transition matrix.

Exercise 3

In a physical space, there is a site and particles are arriving at the site. At time moment t = 0, the site is vacant. At moments t = 1,2, ..., exactly one particle arrives with a probability of p₁. If the site is vacant, the particle occupies the site. If the site is occupied, arriving particles do not enter and are not taken into account. Once a particle occupied the site, at each future moment, it leaves the site with a probability of p₂ and remains there with probability 1− p₂, independently of the prehistory and of whether a new particle has arrived. If at a moment t, a particle left the site and a new particle has arrived, the latter occupies the site.
In one of possible interpretations, the site is a server (say, an auto mechanic) and the particles are customers. In our model, if the server is busy, the customers leave the site. However, there are other possible interpretations. The particles may be electrons, and the site is an orbit (more precisely, a suborbit). The point is that not any electron may go to a suborbit that is already occupied; we skip details.
Let X_t be the number of particles in the site (so, X_t = 0 or 1). Show that X_t is a Markov chain and write its transition matrix.