We consider the approximation of absolutely continuous invariant measures (ACIM's) of systems defined by random compositions of piecewise monotonic transformations. Convergence of Ulam's finite approximation scheme in the case of a single transformation was dealt with by Li. We extend Ulam's construction to the situation where a family of piecewise monotonic transformations are composed according to either an iid or Markov law, and prove an analogous convergence result. In addition, we obtain a convergence rate for our approximations to the unique ACIM, and provide rigorous bounds for the $L^1$ error of the Ulam approximation.