The following commit has made it impossible to remove a cachepool separately from a cache LV (which should result in a conversion of the origin back to a non-cached LV).

8a60cbcf45c553dfc168552cf9af3059359ec95d is the first bad commit
commit 8a60cbcf45c553dfc168552cf9af3059359ec95d
Date:   Tue Mar 11 23:07:41 2014 +0100

    thin: always activate and deactive pool when creating
    
    When we create thin-pool we have used trick to keep
    volume active, but since we now support TEMPORARY flag,
    we could just localy active & deactive metadata LV,
    and let the thinpool through normal activation process.


The problem only seems to happen if I create an origin LV first and then the cache-pool LV, like this:
# lvcreate -L 10G -n lv vg
# lvcreate --type cache -L 1G -n lv_cachepool vg/lv
# lvremove -ff vg/lv_cachepool

Before the above commit, the results looked like this:
# lvcreate -L 10G -n lv vg
  Logical volume "lv" created
# lvcreate --type cache -L 1G -n lv_cachepool vg/lv
  Logical volume "lvol0" created
  Logical volume "lv" created
# lvremove -ff vg/lv_cachepool
  Flushing cache for lv
  6 blocks must still be flushed.
  0 blocks must still be flushed.
  Logical volume "lv_cachepool" successfully removed

After, it looks like this:
# lvcreate -L 10G -n lv vg
  Logical volume "lv" created
# lvcreate --type cache -L 1G -n lv_cachepool vg/lv
  Logical volume "lvol0" created
  Logical volume "lv" created
# lvremove -ff vg/lv_cachepool
  Command failed with status code 5.

I think part of the reason this wasn't caught sooner was because of this commit, which took out the test that creates cache LVs in this way followed by the remove of the cache-pool first:
commit c95d43b28c5f89e1d96f2b16805ecd74eb4b02fb
Date:   Tue Apr 1 17:52:29 2014 +0200

    tests: update lvcreate-cache