Looking at

# cat /usr/lib/python3.9/site-packages/ansible_core-2.13.3-py3.9.egg-info/requires.txt
jinja2>=3.0.0
PyYAML>=5.1
cryptography
packaging
resolvelib<0.9.0,>=0.5.3

and

# ls /usr/lib/python3.9/site-packages/ansible/_vendor/
__init__.py  Jinja2-3.1.2-py3.9.egg-info  MarkupSafe-2.1.0-py3.9.egg-info  packaging-20.4-py3.9.egg-info  pyparsing-2.4.7-py3.9.egg-info  resolvelib
jinja2       markupsafe                   packaging                        __pycache__                    pyparsing.py                    resolvelib-0.5.4-py3.9.egg-info

I guess this also applies to the other requirements?

AFAIK requirements in the requires.txt are generated automatically from python package requirements so that file doesn't need to be updated if some dependencies are bundled or not.

I'm more curious about the pkg_resources usage in the callback.

From what I know, pkg_resources is discouraged in flavor or builtin importlib.metadata and importlib.resources

In this situation you could switch from

- from pkg_resources import iter_entry_points
+ from importlib.metadata import entry_points

and adapt the rest of the code.

With a quick test, I don't have the vendored dependencies issue with importlib.metadata.

[1] https://docs.python.org/3/library/importlib.metadata.html#entry-points

(In reply to Dimitri Savineau from comment #2)
> AFAIK requirements in the requires.txt are generated automatically from
> python package requirements so that file doesn't need to be updated if some
> dependencies are bundled or not.

Yes, this file is autogenerated. Yet it (IMHO) should still reflect the (external) dependencies of the installed software.
Given that resolvelib (and others) are vendored in the Ansible RPM (but not if you install Ansible from e.g. PyPI), the contents of that file should also reflect that you removed the external dependency on resolvelib.

> I'm more curious about the pkg_resources usage in the callback.
> 
> From what I know, pkg_resources is discouraged in flavor or builtin
> importlib.metadata and importlib.resources
> 
> In this situation you could switch from
> 
> - from pkg_resources import iter_entry_points
> + from importlib.metadata import entry_points
> 
> and adapt the rest of the code.

That does sound correct to me, but it's sadly never as easy as that.
The above example was just that, an example.

The real issue I am facing is with the community.general.opentelemetry callback, which in turn depends on the opentelementry library and *that* uses pkg_resources.
The full traceback then reads like this:

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/ansible/cli/__init__.py", line 601, in cli_executor
    exit_code = cli.run()
  File "/usr/lib/python3.9/site-packages/ansible/cli/playbook.py", line 143, in run
    results = pbex.run()
  File "/usr/lib/python3.9/site-packages/ansible/executor/playbook_executor.py", line 119, in run
    self._tqm.load_callbacks()
  File "/usr/lib/python3.9/site-packages/ansible/executor/task_queue_manager.py", line 185, in load_callbacks
    plugin = callback_loader.get(c, class_only=True)
  File "/usr/lib/python3.9/site-packages/ansible/plugins/loader.py", line 825, in get
    return self.get_with_context(name, *args, **kwargs).object
  File "/usr/lib/python3.9/site-packages/ansible/plugins/loader.py", line 845, in get_with_context
    self._module_cache[path] = self._load_module_source(name, path)
  File "/usr/lib/python3.9/site-packages/ansible/plugins/loader.py", line 810, in _load_module_source
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/lib/python3.9/site-packages/ansible_collections/community/general/plugins/callback/opentelemetry.py", line 103, in <module>
    from opentelemetry import trace
  File "/home/user/.local/lib/python3.9/site-packages/opentelemetry/trace/__init__.py", line 87, in <module>
    from opentelemetry import context as context_api
  File "/home/user/.local/lib/python3.9/site-packages/opentelemetry/context/__init__.py", line 22, in <module>
    from pkg_resources import iter_entry_points
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3239, in <module>
    def _initialize_master_working_set():
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3222, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3251, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 567, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 884, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 770, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'resolvelib<0.9.0,>=0.5.3' distribution was not found and is required by ansible-core

And I bet there are many more libraries still using pkg_resources, which, while deprecated, ought to still work.

> The real issue I am facing is with the community.general.opentelemetry callback, which in turn depends on the opentelementry library and *that* uses pkg_resources.

The main issue I have with this is that the ansible-core support scope in RHEL is limited to the rhel-system-roles RPM package (which doesn't use pkg_resources) so the community.general collection isn't supported.

That said, I'll discuss with the upstream core team to see if this is a RPM related issue, an issue with the way the callbacks are loaded or other.

FYI, ansible loads the path to vendored dependencies [1][2] so from the custom callback you have access to the vendored dependencies.

The issue is likely with the pkg_resources usage as it's not using the ansible vendoring path.

[1] https://github.com/ansible/ansible/blob/stable-2.13/lib/ansible/__init__.py#L22-L23
[2] https://github.com/ansible/ansible/blob/stable-2.13/lib/ansible/_vendor/__init__.py

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (ansible-core bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2762