Bug 607994 - Tests requiring a tty fail in Beaker
Tests requiring a tty fail in Beaker
Status: CLOSED CURRENTRELEASE
Product: Beaker
Classification: Community
Component: beah (Show other bugs)
0.5
All Linux
medium Severity medium (vote)
: 0.5.52
: ---
Assigned To: Marian Csontos
: Regression
Depends On:
Blocks: 593365
  Show dependency treegraph
 
Reported: 2010-06-25 07:27 EDT by Petr Šplíchal
Modified: 2016-07-12 00:31 EDT (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-08-02 01:12:42 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Petr Šplíchal 2010-06-25 07:27:14 EDT
Description of problem:

If the task contains a an application which needs a tty, it fails.
For example running simple "top -m -n 1" in Beaker results in the
following error:

	top: failed tty get

An example job:

    https://beaker.engineering.redhat.com/jobs/5194

Works fine in RHTS

    http://rhts.redhat.com/cgi-bin/rhts/test_log.cgi?id=14011053

Version-Release number of selected component (if applicable):
0.5.45
Comment 1 Marian Csontos 2010-06-25 08:51:35 EDT
The problem is RHTS is using /dev/console as stdin/stdout when running tasks.

Though I agree we should try to keep backwards compatibility, I am not sure it is wise to introduce that kind of dependency.

Our tests should be able to run outside of beaker and /dev/console might not be the best choice.
I have tried to use PTY but other tasks were broken.
/dev/null is being used now which seems to work just fine in most cases.

In this case I suggest to "fix" the test.

I noticed other invocations of top are using FakeTTY anyway. Why not using in consistently everywhere?
Comment 2 Kevin Baker 2010-06-25 09:36:32 EDT
Lucas,

we heard through the grapevine that you may have confronted this issue before. Could you talk with Marian and see if your knowledge may help?

thanks
Kev
Comment 3 Petr Šplíchal 2010-06-25 09:46:29 EDT
I guess, there will be more tests which here & there expect to
have a tty attached. For example the /CoreOS/tcsh/Sanity/upstream
also fails in Beaker but works fine in RHTS:

    https://beaker.engineering.redhat.com/jobs/5267
    http://rhts.redhat.com/cgi-bin/rhts/jobs.cgi?id=164294

Not sure this is exactly the tty problem, but I guess it might be
related. Sometimes it can be difficult to tweak upstream test
suites into our tty-less environment.

FakeTTY helps, but running a command through it unfortunately does
not correctly propagate the exit status, which is quite unpleasant
"feature" for testing purposes and must be always work-around-ed.
Comment 4 Marian Csontos 2010-06-28 08:49:43 EDT
I will look into this today.
Comment 5 Lucas Meneghel Rodrigues 2010-06-28 12:19:39 EDT
First of all, I am sorry for taking this long to answer the request :( I was looking at autotest's implementation of subprocess execution to answer this more properly.

As we can see here:

http://autotest.kernel.org/browser/trunk/client/common_lib/utils.py#L64

Autotest's implementation uses subprocess.PIPE as stdin when no specific string is provided, so in the vast majority of the cases, things will run with a pipe as stdin, hence, the commands mentioned on the bug will work under autotest.

After thinking about the problem, it seems to me that the test should be fixed, as Marian pointed out - just make sure that the particular commands are called with /dev/console as stdin on those particular command executions.

I hope this was of some help.
Comment 6 Marian Csontos 2010-06-28 18:23:47 EDT
Hi Petr,
with /dev/console it works better, but I still see two failures in the tcsh test suite.

And I have not run more tests to see if it causes other regressions.

I have added additional log file to be uploaded into test and will retry...

Any help/insight as to what's causing these regressions is appreciated.
Comment 7 Marian Csontos 2010-06-29 07:45:13 EDT
Petr, are you familiar with tcsh test suite? I have tried but do not understand a lot... Could you please have a look at these failures:

https://beaker.engineering.redhat.com/logs/2010/42/5442/9488/106524///testsuite.log

Any idea what's wrong there?
Comment 8 Marian Csontos 2010-06-29 08:39:54 EDT
Thanks for the comment Lucas. If we were building something from scratch, I would definitely go with you recommendation and use /dev/null as default. But as we have a bunch of written tests and compatibility is the highest goal for them.

I will evaluate /dev/console as the default for rhts tasks and will use /dev/null for any future integration work where we are not bound by legacy.
Comment 9 Petr Šplíchal 2010-07-14 08:24:53 EDT
(In reply to comment #5)

> After thinking about the problem, it seems to me that the test
> should be fixed, as Marian pointed out - just make sure that the
> particular commands are called with /dev/console as stdin on
> those particular command executions.

Some of our tests run under regular user to be near real scenario
as much as possible. Using /dev/console as stdin in such cases
results in "permission denied" as /dev/console is owned by root.
Comment 10 Petr Šplíchal 2010-07-14 10:12:47 EDT
(In reply to comment #7)
> Petr, are you familiar with tcsh test suite? I have tried but do
> not understand a lot... Could you please have a look at these
> failures:
> 
> https://beaker.engineering.redhat.com/logs/2010/42/5442/9488/106524///testsuite.log
> 
> Any idea what's wrong there?    

Regarding the failures in the tcsh upstream test suite: Vita
Crhonek might know more, adding him to CC. Vita, could you have a
look at the test suite results above? The test works fine when run
manually but fails in Beaker. Could that be related to the fact,
that no tty is available when run by test harness?
Comment 11 Kevin Baker 2010-07-14 10:31:42 EDT
Hi,

I think breaking compatibilty may be the smartest move here. In other words: fix the tests to work with the new harness. 

How much work would be involved on Dev/QE's side?
Comment 12 Petr Šplíchal 2010-07-19 09:13:57 EDT
Ok, so what's the suggested solution for fixing the tty-dependent
tests? Provided that:

    1) we cannot use /dev/console redirection when testing under a
    regular user because of permission denied (see comment #9)

    2) using python's fake terminal hack results in discarding
    application's exit code which should be checked (comment #3)

Any idea? I don't see any nice solution here...
Comment 13 Lucas Meneghel Rodrigues 2010-07-19 15:03:41 EDT
Beaker can use multiple test harnesses, so maybe it can use autotest to execute tcsh's test suite? Please point me out to the test suite so I can try it here and let you know the results...

If what I am proposing is not doable, please bear with me, I am not intimate with the internals of RHTS/beaker.
Comment 15 Marian Csontos 2010-07-23 08:51:45 EDT
After discussion with Petr, the main problem is python's pty.spawn not returning a sensible value (None).

I have a wrapper (using modified pexpect) which overcomes this problem. Works fine with stdin/stdout being a file or a pipe (unlike pexpect's original interact). I will try to get (at least parts of) it upstream.
Comment 16 Lucas Meneghel Rodrigues 2010-07-23 10:00:44 EDT
I am not sure how active pexpect is being maintained, so it might be tough to get the modifications upstream, but it is certainly worth a try.
Comment 17 Marian Csontos 2010-07-26 03:09:51 EDT
The code is not that difficult, so the wrapper may be standalone one day.
But for now having pexpect it in RHEL{4,5,6} should be enough...
Comment 18 Marian Csontos 2010-07-27 11:34:23 EDT
It looks fine: https://beaker-stage.app.eng.bos.redhat.com/jobs/1246
I will polish it tomorrow and push the tasks to beaker.
Comment 19 Marian Csontos 2010-07-30 09:41:40 EDT
The wrapper fixed top's test suite.

tcsh results have seen some improvement too, but there are still 3 failures (against 5 without PTY wrapper).

There is one failure in common with RHTS.
And remaining 2 failures are not related to TTY.
So far I was unable to get rid of them: from the research I have done it looks like it could be related to nohup.
Comment 20 Marian Csontos 2010-07-30 18:46:51 EDT
Modified task works fine on RHTS as well. Submitted to both RHTS and Beaker.

Note You need to log in before you can comment on or make changes to this bug.