A celebrated result by Jordan, Kinderlehrer and Otto shows that the heat flow on Euclidean space can be viewed as a gradient flow of the entropy in Wasserstein space. In recent years, analog gradient flow characterizations have been obtained in a series of settings, including the heat flow generated by discrete Laplace operators and the time evolution of finite-dimensional open quantum systems. The distinctive common feature of all these evolution equations is Markovianity, i.e., positivity and total mass of the initial value are preserved. For a given (quantum) Markov semigroup, a transport metric on the space of probability measures is constructed, using a Benamou-Brenier-type formula. If the semigroup satisfies a certain gradient estimate, it is shown that the entropy is convex along geodesics for this transport metric and the orbits of the semigroup are curves of steepest descent for the entropy satisfying an evolution variational inequality.