I can make a PR for either option if there is agreement :)

The idea of the original Incus packaging was that the incus-agent is a package that would pull in all the dependencies for VM support so that no separate empty package that you suggested is required. 

What exactly is your use case for wanting to install this individually? In a regular Incus setup the agent is injected by the hypervisor and no separate package must be installed in the VM.

In Gentoo the Incus agent on the hypervisor is even installed into /usr/libexec/incus/agent together with an agent binary for Windows. In this case the dependency to the incus package becomes obvious. I quite like this approach but didn't have time to implement and test it last weekend.

This bug was duplicated by #2456888, where other package maintainers viewed this matter less critically than I did and simply removed the dependency from the agent package, releasing a new package update. Strictly speaking, this bug is therefore "fixed" for now.

What I fail to understand is the use case for someone wanting to install the `incus-agent` package independently of the `incus` daemon.

The agent implementation is tightly coupled to the `incusd` implementation [1], and its purpose of being installed in `/usr/bin` is not that it is run from there directly, but rather that it is picked up from there by `incusd` [2]. As already mentioned in the previous comment, I find the second approach described in the linked packaging recommendations more favorable, as it would also allow shipping a Windows agent — which is currently missing from the Fedora package — and would avoid the confusion that, in my opinion, led to this bug report in the first place. If the agent is installed via a separate package into the VM, a user would first need to disable agent injection via `incusd`, then manually create a corresponding systemd service [3], and also keep the hypervisor and agent versions manually aligned.

If there are no objections, I would prepare a revert of the change introduced in `6.23-3` [4] and move the agent binary to a dedicated `INCUS_AGENT_PATH`. In the past I have always treated the `incus-agent` package as wrapper for "incusd + VM support" [5]. If people prefer an explicit wrapper package for this use case I am fine with that too but personally I consider it unnecessary overhead. Either way, the incus-agent package as was (and still is) should not be confused with a standalone agent setup.

If there is a good reason to ship the agent binary independently of `incus`, we might want to create a separate `incus-agent-standalone` package instead. If the use case for that were clearly defined then it would also be possible to create a working experience out of the box which requires a bit more than just the agent binary alone as can be seen in the corresponding Debian package [6].

[1]: https://github.com/lxc/incus/issues/263
[2]: https://linuxcontainers.org/incus/docs/main/packaging/#incus-agent-binaries
[3]: https://github.com/lxc/incus/blob/main/internal/server/instance/drivers/agent-loader/systemd/incus-agent.service
[4]: https://src.fedoraproject.org/rpms/incus/pull-request/11#
[5]: https://github.com/ganto/copr-lxc4/wiki#incus
[6]: https://packages.debian.org/trixie/amd64/incus-agent/filelist

OK, I had not realized that there was no interface stability guarantee for the agent which is a really bad idea. This means that you must use the agent binary provided by the hypervisor in the VM. This does not really work well with systems that want to only run verified binaries only and not something provided by the hypervisor. The main use case I was starting from is support for Fedora CoreOS.

On all other hypervisor (qemu-quest-agent, aws/gcp/azure), you install the client in the VM and that's it. From what I can gather from https://linuxcontainers.org/incus/docs/main/howto/instances_create/#install-the-incus-agent-into-virtual-machine-instances, you have to use the pre-prepared images from https://images.linuxcontainers.org/ (again, this is unacceptable and a really bad idea: how do I know what's in those and trust those?) or you have to run an install script from a virtual CD-ROM drive, which is also unacceptable as this script may do anything and can not reasonably be audited head of time.

The approach I was expecting is that we would provide a package that only includes the agent so that it can be installed as needed into the images or into a sysexts for image based/immutable systems such as Fedora CoreOS.

I agree with Timothée, this is a bad design for a number of reasons.  In addition to the points he raised, I'll emphasize that the standard expectation across the virtualization ecosystem is that agent subpackages are for installation in guests.  That's what led to this bug as well as bug 2456888 getting filed.  The suggested idea for incus-agent-standalone is what people expect incus-agent to be.  If the agent binary must be installed on the host for injection purposes, then what is even the point of splitting the agent binary into a separate subpackage?

Hello, I'm the reporter of 2456888 and I wanted to share my perspective here.

@Reto
> What I fail to understand is the use case for someone wanting to install the `incus-agent` package independently of the `incus` daemon.

Incus is a Debian-first project, being a fork of Canonical's LXD[0] . The experience on Debian is that the `incus-agent` is a standalone package meant to be installed on virtual guests:

```
apt info incus-agent

Description: Incus guest agent
 This package provides an agent to run inside Incus virtual machine guests.
```

And as Carl pointed out, "the standard expectation across the virtualization ecosystem is that agent subpackages are for installation in guests." I would argue that we should follow the principle of least surprise and align with Debian and general virt experiences unless there is a major advantage. I also read the packaging recommendations you linked but I did not see any suggestions that that agent packages should depend on the entire Incus stack being installed. 

Even if you managed to convince everyone on this bug report that "the incus-agent package as was (and still is) should not be confused with a standalone agent setup," others will not understand this and more bug reports are likely to appear.


```
If the agent is installed via a separate package into the VM, a user would first need to disable agent injection via `incusd`, then manually create a corresponding systemd service [3]
```

I'm not sure I follow. The Debian package provides the systemd unit file, why wouldn't that be part of the Fedora package?


@Timothée

```
 you have to use the pre-prepared images from https://images.linuxcontainers.org/ (again, this is unacceptable and a really bad idea: how do I know what's in those and trust those?) 
```
Linuxcontainers.org==Incus. Incus is a very "batteries included" platform. Besides providing its own images, it does some unusual things for virtual machine manager such as supporting OCI (docker images)[2] and proxies [3]. 

But you don't have to use their images, I'm using my own custom Fedora image. I want to be able to use my custom images without doing complex orchestrations or installing an entire virtual machine manager just for the agent, that's why I filed basically the same bug as you ;) .

```
or you have to run an install script from a virtual CD-ROM drive, which is also unacceptable as this script may do anything and can not reasonably be audited head of time.
```

Yes, that's another thing we can avoid by having packages.

Thanks again everyone for your time, I started a thread on the Incus forums encouraging people to give their feedback here[4].


[0] https://discuss.linuxcontainers.org/t/introducing-incus/17781
[1] https://github.com/lxc/incus/issues/2800
[2] https://blog.simos.info/running-oci-images-i-e-docker-directly-in-incus/
[3] https://linuxcontainers.org/incus/docs/main/reference/devices_proxy/
[4] https://discuss.linuxcontainers.org/t/should-the-fedora-incus-agent-package-require-the-entire-incus-stack-to-be-installed-let-your-voice-be-heard/26576

@Timothée
Ok, the reason that there seems to be some virtualization ecosystem standard around agent packages and that Incus should follow this standard also to improve the security footprint make sense to me. As long as it's technically feasible. Being familiar with the Linux ecosystem for the last 2 decades I have to admit that I've always ignored those "agents" or even considered them a security threat exactly because they were often not packaged properly. Still what I cannot change is the current design choice where the agent is injected via incusd. It was made by the authors and should be addressed there. I was simply trying to come up with a package layout that reflected the technical dependencies and that was easy to use.

@Bryan
After my recent response I also had a look at the Debian package and in the meantime I think it makes sense to copy its layout. I'm packaging LXD and later Incus since nearly 10 years [1]. Because of that I don't fully agree it being a "Debian-first" project. It was an Ubuntu-first project. Originally there was an Ubuntu lxd dpkg that I copied. Later on, when virtualization support was introduced it was only available via Snap and so I had to come up with my own layout. At that time also Debian has chosen a similar layout [2] where they even referenced my "Fedora package" layout. However they adjusted it slightly to not be a "wrapper" package depending on QEMU. As I haven't used LXD/Incus on Debian before I don't know exactly how they pull in the necessary dependencies. However even today with the Debian "incus-agent" package [3] it's still documented as "It has to be installed on the Incus host if you want to allow agent injection capability when creating a virtual machine."

Long story short I think it make sense to align with Debian and I'll investigate if or how they made the agent package installable in a VM. Unfortunately upstream is not helping much in regard of systemd integration as they maintain yet another package layout [4] and obviously fully depend on agent injection. 

[1]: https://github.com/ganto/copr-lxd
[2]: https://salsa.debian.org/go-team/packages/lxd/-/commit/8b9c0980fc3c6e8d39b8916ae64364e9acd51382
[3]: https://salsa.debian.org/go-team/packages/incus/-/blob/debian/sid/debian/control#L300
[4]: https://github.com/zabbly/incus/blob/daily/debian/control

I guess a nice usability would be that either you install agent on the host then it's injected via incusd or you omit it on the host and install it in the VM and in both cases you would have the same functionality.

I'm engaging with upstream to figure out what to do as part of preparing the 7.0 update.

@Reto thanks for taking the time to package all this stuff and helping us understand the other side of the debate. It's just my opinion, but I think it's fine to have an agent-only package for the guest without worrying about the host requirements, in the same way you might separate NFS client and server packages (You could make the server packages depend on the client without the client depending on the server). 

I also think the agent injection feature might only work with LXC as opposed to VMs? I believe that you have to install the agent with the virtual CD-ROM drive as Timothée mentioned above. VMWare, XenServer etc follow that pattern where the hypervisor exposes a virtual CD-ROM to the guest for installing guest tools. It works, but agent installation via OS package is more convenient. I'm still just learning Incus so I may have some bad assumptions here, feel free to respond if I missed anything.

For containers (LXC) the agent is not necessary as these are simply namespaced processes. For a privileged enough process on the host system (e.g. incusd) it's always possible to enter the namespace of a container and either get some information out or spawn a process e.g. an administration shell.

The agent is only necessary for VMs where you have to "bridge" the OS kernel. The mechanism is as Timothée says that the VM is copying the agent from a 9p config drive or virtual cdrom into the VM [1]. The images from https://images.linuxcontainers.org/ do this via a systemd service by default already. That was what I was referring to as injecting the agent. If you are interested how they are built that's all open and transparent at [2].

Of course you're right if you bring your own OS image and want to have all software on board already an agent package is what you need.

[1]: https://github.com/lxc/distrobuilder/blob/main/generators/incus-agent.go
[2]: https://github.com/lxc/lxc-ci/tree/main/images