We observe that the periodic variations in spin-down rate and beam-width of the radio pulsar PSR B1828-11 are getting faster. In the context of a free precession model, this corresponds to a decrease in the precession period $P_{\mathrm{fp}}$. We investigate how a precession model can account for such a decrease in $P_{\mathrm{fp}}$, in terms of an increase over time in the absolute biaxial deformation ($|\epsilon_{\mathrm{p}}|{\sim}10^{-8}$) of this pulsar. We perform a Bayesian model comparison against the 'base' precession model (with constant $\epsilon_{\mathrm{p}}$) developed in Ashton et al (2016), and we obtain decisive odds in favour of a time-varying deformation. We study two types of time-variation: (i) a linear drift with a posterior estimate of $\dot{\epsilon}_{\mathrm{p}}{\sim}10^{-18}\,\mathrm{s}^{-1}$ and odds of $10^{75}$ compared to the base-model, and (ii) $N$ discrete positive jumps in $\epsilon_{\mathrm{p}}$ with very similar odds to the linear $\epsilon_{\mathrm{p}}$-drift model. The physical mechanism explaining this behaviour is unclear, but the observation could provide a crucial probe of the interior physics of neutron stars. We also place an upper bound on the rate at which the precessional motion is damped, and translate this into a bound on a dissipative mutual friction-type coupling between the star's crust and core.