Bug 1574624 - virsh list unicode column layout confusion
Summary: virsh list unicode column layout confusion
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Virtualization Tools
Classification: Community
Component: libvirt
Version: unspecified
Hardware: All
OS: All
urgent
high
Target Milestone: ---
Assignee: Libvirt Maintainers
QA Contact:
URL:
Whiteboard: LibvirtFirstBug
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-05-03 17:01 UTC by Dr. David Alan Gilbert
Modified: 2018-09-24 11:26 UTC (History)
7 users (show)

Fixed In Version: libvirt-4.8.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-09-24 10:24:58 UTC


Attachments (Terms of Use)

Description Dr. David Alan Gilbert 2018-05-03 17:01:54 UTC
Description of problem:
With a VM that has a unicode name, the 'state' column is incorrectly aligned

Version-Release number of selected component (if applicable):
libvirt-client-4.1.0-2.fc28.x86_64
(Also seen on rhel7)

How reproducible:
100%

Steps to Reproduce:
1. Create a VM with a multibyte unicode character in the name (e.g. an alpha symbol, or something more fun like '🐧⃞ '
2. virsh list --all
3.

Actual results:
The 'running' state of the unicode named domain is too far to the left, probably because it's counting bytes not characters

Expected results:
State is still aligned

Additional info:

Comment 1 Daniel Berrangé 2018-05-04 09:22:58 UTC
The core problem is probably that we're using strlen() to check length and thus decide padding, which simply counts bytes. We need to count printable unicode characters - mbstowcs() probably gets us the solution we need, or very close to it.

Comment 2 Dr. David Alan Gilbert 2018-05-04 09:41:36 UTC
(In reply to Daniel Berrange from comment #1)
> The core problem is probably that we're using strlen() to check length and
> thus decide padding, which simply counts bytes. We need to count printable
> unicode characters - mbstowcs() probably gets us the solution we need, or
> very close to it.

I'm not sure if mbstowcs is the right one;  certainly any utf8 I throw at it returns -1.

Comment 3 Daniel Berrangé 2018-05-04 10:01:22 UTC
Yes, it seems I get that too. Bizarrely mbstowcs() does not work if your locale is set to UTF-8, but if you blank out locale setlocale(LC_CTYPE, ""); it then does the right thing with utf8 strings. Blanking locale in this way is not acceptable, though, so we need to find something better...

It is possible to count the number of utf-8 characters using this:

size_t utf8len(char *s)
{
    size_t len = 0;
    for (; *s; ++s) if ((*s & 0xC0) != 0x80) ++len;
    return len;
}

IIUC, this should be the same as what mbstowcs() would report. 

Note that is not the same as calculating the printable output length though, because of zero-width characters and RTL text, combining characters, etc :-( It would still be somewhat better that what we have today though.

BTW, I was wrong about us using strlen, we in fact rely on printf format string padding

                vshPrint(ctl, " %-5s %-30s %-10s %-20s\n", id_buf,
                         virDomainGetName(dom),
                         state == -2 ? _("saved")
                         : virshDomainStateToString(state),
                         title);

So whatever solution we choose, we'll need to replace that. Instead of trying to print the entire line at once, we'll need to print each field separately, calculating padding using some unicode aware function.

Might be a bit ambitious, but I'm labelling this a LibvirtFirstBug, since it is non-urgent and fairly self-contained work.

Comment 4 Daniel Berrangé 2018-06-06 08:48:38 UTC
Another case where we get column layout wrong but for non-unicode related reasons:

https://bugzilla.redhat.com/show_bug.cgi?id=1584630

We need to come up with some general purpose internal API for printing out tables of data, and use that everywhere in virsh to get alignment right, instead of printing tables manually with each command.

Comment 5 Richard W.M. Jones 2018-08-08 15:11:41 UTC
Not sure if it handles unicode properly, but:
https://github.com/karelzak/util-linux/tree/master/libsmartcols

Comment 6 Richard W.M. Jones 2018-08-08 15:15:11 UTC
I think it doesn't handle unicode properly, but should be fixed
to do so.  It seems anyway easier to use libsmartcols and fix it
rather than trying to invent something new.  It is packaged separately
in RHEL.

Comment 7 Daniel Berrangé 2018-08-08 15:18:38 UTC
Main criteria would be platform portability - would need it available on OS-X (homebrew),  mingw in Fedora, FreeBSD, and of course the Linuxes.

Comment 8 Karel Zak 2018-08-08 18:13:16 UTC
We (util-linux upstream) compile libsmartcols on

 Apple LLVM version 7.3.0 (clang-703.0.31)
 Target: x86_64-apple-darwin15.6.0

see https://travis-ci.org/karelzak/util-linux/jobs/413533274

It seems Debian has binary packages for kFreeBSD
https://packages.debian.org/sid/libsmartcols1

Not sure about mingw.

Comment 9 Michal Privoznik 2018-08-09 08:55:59 UTC
Bummer, we've already have some patches that implement our own table (the patches are still being actively worked on).

https://github.com/simmmon/libvirt/commits/tableAPI

Comment 10 Simon Kobyda 2018-08-10 09:21:04 UTC
Patch proposed upstream:
https://www.redhat.com/archives/libvir-list/2018-August/msg00549.html

Comment 11 Simon Kobyda 2018-09-24 10:15:23 UTC
Patches pushed upstream:
https://www.redhat.com/archives/libvir-list/2018-September/msg01184.html

Comment 12 Michal Privoznik 2018-09-24 11:26:06 UTC
bfdd20c5a9 virt-admin: Implement vshTable API to server-list and client-list
95b29fc222 virsh: Implement vshTable API to vol-list
79eec7992b virsh: Implement vshTable API to pool-list
3072ded354 virsh: Implement vshTable API to vcpupin, iothreadinfo, domfsinfo
063509a193 virsh: Implement vshTable API to domiflist
075dd1185d virsh: Implement vshTable API to domblklist
2979bbfb0f virsh: Implement vshTable API to domblkinfo
b9057c639f virsh: Set up cmdDomblkinfo() and cmdDomblkinfoPrint() for vshTable API implementation
b0b1ed2f7b virsh: Implement vshTable API to snapshot-list.
c2b9cf6733 virsh: Implement vshTable API to nwfilter-list and nwfilterbinding-list
cf12efe088 virsh: Implement vshTable API to secret-list
71029ef588 virsh: Implement vshTable API to net-list and net-dhcp-leases
0396cf5336 virsh: Implement vsh-table to iface-list

v4.7.0-226-gbfdd20c5a9


Note You need to log in before you can comment on or make changes to this bug.