In a Markov chain, the matrix (or matrices) are similar to the matrix representation of a graph (without brackets):

    a b c d
  a 1 0 1 0
  b 1 1 0 0
  c 0 0 1 1
  d 1 0 0 1

Written another way, the above graph has these directed edges:

  a -> a, a -> c
  b -> a, b -> b
  c -> c, c -> d
  d -> a, d -> d

In a Markov chain, the possible state transitions (graph edges) are represented in the same way, but instead of a 1 or 0 (connects or doesn't connect) a probability is assigned to each possible transition.

https://en.wikipedia.org/wiki/Examples_of_Markov_chains - Several examples with a mix of prose, graphical presentation, and the matrix representation.