Bug 1523459

(In reply to Ping Li from comment #0)
> Description of problem:
> After install os on the image encrypted by LUKS, fail to dump the metadata
> of the image using human format. But it could be successful using json
> format. It will be strange that human format and json format work
> differently. As the human format is the default, So I think we should
> support it.

The two outputs are different in what they report.  The JSON output intentionally conveys more information than the human output.  The human output is not supposed to be machine-parsed, so we could theoretically change it to improve the amount of information it outputs, but I'm not sure if that is a wise idea, or if we should just mark this as won't fix.


> Steps to Reproduce:
> 1. create an image encrypted with luks
> # qemu-img create --object secret,id=sec0,data=backing -f qcow2 -o
> encrypt.format=luks,encrypt.key-secret=sec0 base.qcow2 20G
> Formatting 'base.qcow2', fmt=qcow2 size=21474836480 encrypt.format=luks
> encrypt.key-secret=sec0 cluster_size=65536 lazy_refcounts=off
> refcount_bits=16
>      
> 2. Dump the metadata
> # qemu-img map --object secret,id=sec0,data=backing --image-opts
> driver=qcow2,encrypt.key-secret=sec0,file.filename=base.qcow2
> Offset          Length          Mapped to       File
>      
> # qemu-img map --object secret,id=sec0,data=backing --image-opts
> driver=qcow2,encrypt.key-secret=sec0,file.filename=base.qcow2 --output=json
> [{ "start": 0, "length": 21474836480, "depth": 0, "zero": true, "data":
> false}]

At this point, the difference matches non-encrypted images: the JSON output describes holes, but the human output only describes literal mappings (of which there are none).

>      
> 3. Install os on the image.
> 
> 4. Dump the metadata ---> failed to dump with human format
> # qemu-img map --object secret,id=sec0,data=backing --image-opts
> driver=qcow2,encrypt.key-secret=sec0,file.filename=base.qcow2
> Offset          Length          Mapped to       File
> qemu-img: File contains external, encrypted or compressed clusters.

Here, the output matches reality.  There are encrypted clusters, so we CAN'T tell you what the offsets are mapped to (because reading at that offset will see only the encrypted data, although the human-readable format shows only the offsets that you can directly read without having to decompress or decrypt).

> 
> # qemu-img map --object secret,id=sec0,data=backing --image-opts
> driver=qcow2,encrypt.key-secret=sec0,file.filename=base.qcow2 --output=json
> [{ "start": 0, "length": 65536, "depth": 0, "zero": false, "data": true},

And the JSON output indeed displays more information, because it is describing as much as possible, rather than just the places that can be directly read without decompression or decryption.

> Actual results:
> Fail to dump metadata using human format
> 
> Expected results:
> It should be ok using both human and json format
> 
> Additional info:
> The issue doesn't occur on the image without encrypted.

It also occurs on unencrypted images that use compression.  We can raise the question upstream whether the human output should be more verbose, but it may just be that if you want full information, you have to use the JSON output.

This issue still exists in RHEL-8.0.1-20190325.n.0-ppc64le
kernel-4.18.0-80.el8.ppc64le
qemu-kvm-3.1.0-20.module+el8+2888+cdc893a8

How reproducible:
100%

Steps to Reproduce:
1. create an image encrypted with luks
# qemu-img create --object secret,id=sec0,data=backing -f qcow2 -o encrypt.format=luks,encrypt.key-secret=sec0 base.qcow2 20G 
Formatting 'base.qcow2', fmt=qcow2 size=21474836480 encrypt.format=luks encrypt.key-secret=sec0 cluster_size=65536 lazy_refcounts=off refcount_bits=16

2. Install os on the image.

3. Dump the metadata ---> failed to dump with human format
# qemu-img map --object secret,id=sec0,data=backing --image-opts driver=qcow2,encrypt.key-secret=sec0,file.filename=base.qcow2
Offset          Length          Mapped to       File
qemu-img: File contains external, encrypted or compressed clusters.

# qemu-img map --object secret,id=sec0,data=backing --image-opts driver=qcow2,encrypt.key-secret=sec0,file.filename=base.qcow2 --output=json
[{ "start": 0, "length": 65536, "depth": 0, "zero": false, "data": true},
{ "start": 65536, "length": 983040, "depth": 0, "zero": true, "data": false},
{ "start": 1048576, "length": 177602560, "depth": 0, "zero": false, "data": true},
{ "start": 178651136, "length": 95027200, "depth": 0, "zero": true, "data": false},
{ "start": 273678336, "length": 2686976, "depth": 0, "zero": false, "data": true},
{ "start": 276365312, "length": 265748480, "depth": 0, "zero": true, "data": false},
{ "start": 542113792, "length": 10616832, "depth": 0, "zero": false, "data": true},
{ "start": 552730624, "length": 257818624, "depth": 0, "zero": true, "data": false},
{ "start": 810549248, "length": 327680, "depth": 0, "zero": false, "data": true},
{ "start": 810876928, "length": 267976704, "depth": 0, "zero": true, "data": false},
{ "start": 1078853632, "length": 1245184, "depth": 0, "zero": false, "data": true},
{ "start": 1080098816, "length": 2147418112, "depth": 0, "zero": true, "data": false},
{ "start": 3227516928, "length": 153223168, "depth": 0, "zero": false, "data": true},
{ "start": 3380740096, "length": 65536, "depth": 0, "zero": true, "data": false},
{ "start": 3380805632, "length": 63373312, "depth": 0, "zero": false, "data": true},
{ "start": 3444178944, "length": 65536, "depth": 0, "zero": true, "data": false},
{ "start": 3444244480, "length": 173015040, "depth": 0, "zero": false, "data": true},
{ "start": 3617259520, "length": 4171563008, "depth": 0, "zero": true, "data": false},
{ "start": 7788822528, "length": 195952640, "depth": 0, "zero": false, "data": true},
{ "start": 7984775168, "length": 3211264, "depth": 0, "zero": true, "data": false},
{ "start": 7987986432, "length": 1638400, "depth": 0, "zero": false, "data": true},
{ "start": 7989624832, "length": 393216, "depth": 0, "zero": true, "data": false},
{ "start": 7990018048, "length": 81788928, "depth": 0, "zero": false, "data": true},
{ "start": 8071806976, "length": 4278321152, "depth": 0, "zero": true, "data": false},
{ "start": 12350128128, "length": 168230912, "depth": 0, "zero": false, "data": true},
{ "start": 12518359040, "length": 6422528, "depth": 0, "zero": true, "data": false},
{ "start": 12524781568, "length": 54394880, "depth": 0, "zero": false, "data": true},
{ "start": 12579176448, "length": 4332257280, "depth": 0, "zero": true, "data": false},
{ "start": 16911433728, "length": 544997376, "depth": 0, "zero": false, "data": true},
{ "start": 17456431104, "length": 4016177152, "depth": 0, "zero": true, "data": false},
{ "start": 21472608256, "length": 131072, "depth": 0, "zero": false, "data": true},
{ "start": 21472739328, "length": 2031616, "depth": 0, "zero": true, "data": false},
{ "start": 21474770944, "length": 65536, "depth": 0, "zero": false, "data": true}]

Closing this as wontfix per comment #3; upstream made the intentional decision that the human-readable version only shows direct mappings (and compressed/encrypted images don't have direct mappings) and is constrained to that format due to the choice of column headers implying that the data is already processed from the raw layer into the format matching the columns; while the JSON output shows everything at a lower layer and is not constrained by formatting.