Bug 2173980
| Summary: | The path to the guest agent socket file can become too long and cause problems | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | Jed Lejosne <jlejosne> | |
| Component: | libvirt | Assignee: | Michal Privoznik <mprivozn> | |
| libvirt sub component: | General | QA Contact: | Lili Zhu <lizhu> | |
| Status: | CLOSED MIGRATED | Docs Contact: | ||
| Severity: | high | |||
| Priority: | unspecified | CC: | aasserzo, acardace, bbenshab, bstinson, ibezukh, jdenemar, jhopper, jlejosne, jsuchane, jwboyer, kbidarka, lmen, mprivozn, qixuan.wang, smitterl, virt-maint, yadu, ycui, ymankad | |
| Version: | CentOS Stream | Keywords: | MigratedToJIRA, Triaged, ZStream | |
| Target Milestone: | rc | |||
| Target Release: | --- | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | 2168346 | |||
| : | 2225204 2233744 2233788 (view as bug list) | Environment: | ||
| Last Closed: | 2023-09-22 17:53:31 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 2168346 | |||
| Bug Blocks: | 2225204, 2233744, 2233788 | |||
|
Description
Jed Lejosne
2023-02-28 15:00:37 UTC
The maximum socket path in Linux is 107. In KubeVirt, we can end up with a 108 characters path for guest-agent, like: /var/run/kubevirt-private/libvirt/qemu/channel/target/domain-214-default_rhel82-vm000/org.qemu.guest_agent.0 We need to find a way to either shorten the path or replace it with something else, like a file descriptor. I see couple ways to fix this issue:
1) make the cfg->channelDir configurable so that mgmt apps can skip "/libvirt/qemu/channel/target/" part, or
2) KubeVirt may use shorter VM name,
3) Libvirt may generate even shorter "shortened names" (currently 20 chars), since we're guaranteed to be unique regardless of short name length (due to domain ID in the path).
It was also suggested that libvirt could just pass FD to QEMU. I did not realize it back then, but libvirt's already doing that, e.g.:
-chardev socket,id=charchannel0,fd=32,server=on,wait=off \
-device {"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}
but that won't really fly when libvirtd restarts and needs to reconnect back to the socket. Unless there's a way in QEMU to provide a new fd, in which case libvirt could just use an unnamed socket and tell QEMU the new FD on reconnect.
Reproduce this bug with:
libvirt-9.0.0-7.el9.x86_64
1. Create directory for guest agent socket source:
# mkdir /var/run/kubevirt-private/libvirt/qemu/channel/target/domain-212-default_rhel82-vm000/ -p
2. Change the owner and selinux context of the directory:
# chown qemu:qemu /var/run/kubevirt-private/libvirt/qemu/channel/target/domain-212-default_rhel82-vm000/
# chcon system_u:object_r:qemu_var_run_t:s0 /var/run/kubevirt-private/libvirt/qemu/channel/target/domain-212-default_rhel82-vm000/
3. prepare a guest with the following xml snippet
<channel type='unix'>
<source mode='bind' path='/var/run/kubevirt-private/libvirt/qemu/channel/target/domain-212-default_rhel82-vm000/org.qemu.guest_agent.1'/>
<target type='virtio' name='org.qemu.guest_agent.0'/>
<address type='virtio-serial' controller='0' bus='0' port='2'/>
</channel>
4. start the guest
# virsh start vm1
error: Failed to start domain 'vm1'
error: internal error: UNIX socket path '/var/run/kubevirt-private/libvirt/qemu/channel/target/domain-212-default_rhel82-vm000/org.qemu.guest_agent.1' too long
Additional info:
If I change the last character of the path to '0', i.e., /var/run/kubevirt-private/libvirt/qemu/channel/target/domain-212-default_rhel82-vm000/org.qemu.guest_agent.0
<channel type='unix'>
<source mode='bind' path='/var/run/kubevirt-private/libvirt/qemu/channel/target/domain-212-default_rhel82-vm000/org.qemu.guest_agent.0'/>
<target type='virtio' name='org.qemu.guest_agent.0'/>
<address type='virtio-serial' controller='0' bus='0' port='2'/>
</channel>
libvirt will remove the specified path, and use the default socket path,
<channel type='unix'>
<source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-1-vm1/org.qemu.guest_agent.0'/>
<target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
<alias name='channel0'/>
<address type='virtio-serial' controller='0' bus='0' port='2'/>
</channel>
Hi, Michal
I am curious why the above path "/var/run/kubevirt-private/libvirt/qemu/channel/target/domain-212-default_rhel82-vm000/org.qemu.guest_agent.0" was not removed in cnv. Please help to check if you are available. Thanks very much.
(In reply to Lili Zhu from comment #3) > Hi, Michal > > I am curious why the above path > "/var/run/kubevirt-private/libvirt/qemu/channel/target/domain-212- > default_rhel82-vm000/org.qemu.guest_agent.0" was not removed in cnv. Please > help to check if you are available. Thanks very much. I believe they don't specify the path. For the guest agent, libvirt puts the socket into so called channelTargetDir, which is derived as follows: virQEMUDriverConfig * virQEMUDriverConfigNew(bool privileged, const char *root) { if (root != NULL) { cfg->channelTargetDir = g_strdup_printf("%s/channel/target", cfg->libDir); } else if (privileged) { cfg->channelTargetDir = g_strdup_printf("%s/channel/target", cfg->libDir); } else { cfg->configBaseDir = virGetUserConfigDirectory(); cfg->channelTargetDir = g_strdup_printf("%s/qemu/channel/target", cfg->configBaseDir); } } and since CNV uses qemu:///session the third option is used. In here, virGetUserConfigDirectory() basically returns "$XDG_CONFIG_HOME/libvirt". In other words, this path is autogenerated by libvirt and as the domain ID grows, it also prolongs the socket path. And since the path was long enough to begin with, after couple of iterations, the domain ID is large enough to make the generated path not fit into the limit. Hi Michal, Lili, Can you please update on the bug status? would you pick it for upcoming release? TIA, Igor Patches proposed on the list: https://listman.redhat.com/archives/libvir-list/2023-July/240680.html *** Bug 2225204 has been marked as a duplicate of this bug. *** Merged upstream as: 8abc979bb0 qemu: Move channelTargetDir into stateDir d3759d3674 qemu: Generate shorter channel target paths v9.6.0-47-g8abc979bb0 Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug. This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there. Due to differences in account names between systems, some fields were not replicated. Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information. To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer. You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like: "Bugzilla Bug" = 1234567 In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information. |