Test with podman-1.0.3-1.git9d78c0c.module+el8.0.0.z+3717+fdd07b7c.x86_64. And seems the exit code already updated. So set this to verified, Details:
# podman exec memcached /bin/true; echo $?
unable to exec into memcached: no container with name or ID memcached found: no such container
125
# podman ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
eb4605318818 docker.io/library/busybox:latest sh 8 seconds ago Exited (0) 8 seconds ago unruffled_mirzakhani
# podman exec eb4605318818 /bin/true ; echo $?
cannot exec into container that is not running: container state improper
126
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2019:1956