Bug 607994

The problem is RHTS is using /dev/console as stdin/stdout when running tasks.

Though I agree we should try to keep backwards compatibility, I am not sure it is wise to introduce that kind of dependency.

Our tests should be able to run outside of beaker and /dev/console might not be the best choice.
I have tried to use PTY but other tasks were broken.
/dev/null is being used now which seems to work just fine in most cases.

In this case I suggest to "fix" the test.

I noticed other invocations of top are using FakeTTY anyway. Why not using in consistently everywhere?

Lucas,

we heard through the grapevine that you may have confronted this issue before. Could you talk with Marian and see if your knowledge may help?

thanks
Kev

I guess, there will be more tests which here & there expect to
have a tty attached. For example the /CoreOS/tcsh/Sanity/upstream
also fails in Beaker but works fine in RHTS:

    https://beaker.engineering.redhat.com/jobs/5267
    http://rhts.redhat.com/cgi-bin/rhts/jobs.cgi?id=164294

Not sure this is exactly the tty problem, but I guess it might be
related. Sometimes it can be difficult to tweak upstream test
suites into our tty-less environment.

FakeTTY helps, but running a command through it unfortunately does
not correctly propagate the exit status, which is quite unpleasant
"feature" for testing purposes and must be always work-around-ed.

I will look into this today.

First of all, I am sorry for taking this long to answer the request :( I was looking at autotest's implementation of subprocess execution to answer this more properly.

As we can see here:

http://autotest.kernel.org/browser/trunk/client/common_lib/utils.py#L64

Autotest's implementation uses subprocess.PIPE as stdin when no specific string is provided, so in the vast majority of the cases, things will run with a pipe as stdin, hence, the commands mentioned on the bug will work under autotest.

After thinking about the problem, it seems to me that the test should be fixed, as Marian pointed out - just make sure that the particular commands are called with /dev/console as stdin on those particular command executions.

I hope this was of some help.

Hi Petr,
with /dev/console it works better, but I still see two failures in the tcsh test suite.

And I have not run more tests to see if it causes other regressions.

I have added additional log file to be uploaded into test and will retry...

Any help/insight as to what's causing these regressions is appreciated.

Petr, are you familiar with tcsh test suite? I have tried but do not understand a lot... Could you please have a look at these failures:

https://beaker.engineering.redhat.com/logs/2010/42/5442/9488/106524///testsuite.log

Any idea what's wrong there?

Thanks for the comment Lucas. If we were building something from scratch, I would definitely go with you recommendation and use /dev/null as default. But as we have a bunch of written tests and compatibility is the highest goal for them.

I will evaluate /dev/console as the default for rhts tasks and will use /dev/null for any future integration work where we are not bound by legacy.

(In reply to comment #5)

> After thinking about the problem, it seems to me that the test
> should be fixed, as Marian pointed out - just make sure that the
> particular commands are called with /dev/console as stdin on
> those particular command executions.

Some of our tests run under regular user to be near real scenario
as much as possible. Using /dev/console as stdin in such cases
results in "permission denied" as /dev/console is owned by root.

(In reply to comment #7)
> Petr, are you familiar with tcsh test suite? I have tried but do
> not understand a lot... Could you please have a look at these
> failures:
> 
> https://beaker.engineering.redhat.com/logs/2010/42/5442/9488/106524///testsuite.log
> 
> Any idea what's wrong there?    

Regarding the failures in the tcsh upstream test suite: Vita
Crhonek might know more, adding him to CC. Vita, could you have a
look at the test suite results above? The test works fine when run
manually but fails in Beaker. Could that be related to the fact,
that no tty is available when run by test harness?

Hi,

I think breaking compatibilty may be the smartest move here. In other words: fix the tests to work with the new harness. 

How much work would be involved on Dev/QE's side?

Ok, so what's the suggested solution for fixing the tty-dependent
tests? Provided that:

    1) we cannot use /dev/console redirection when testing under a
    regular user because of permission denied (see comment #9)

    2) using python's fake terminal hack results in discarding
    application's exit code which should be checked (comment #3)

Any idea? I don't see any nice solution here...

Beaker can use multiple test harnesses, so maybe it can use autotest to execute tcsh's test suite? Please point me out to the test suite so I can try it here and let you know the results...

If what I am proposing is not doable, please bear with me, I am not intimate with the internals of RHTS/beaker.

After discussion with Petr, the main problem is python's pty.spawn not returning a sensible value (None).

I have a wrapper (using modified pexpect) which overcomes this problem. Works fine with stdin/stdout being a file or a pipe (unlike pexpect's original interact). I will try to get (at least parts of) it upstream.

I am not sure how active pexpect is being maintained, so it might be tough to get the modifications upstream, but it is certainly worth a try.

The code is not that difficult, so the wrapper may be standalone one day.
But for now having pexpect it in RHEL{4,5,6} should be enough...

It looks fine: https://beaker-stage.app.eng.bos.redhat.com/jobs/1246
I will polish it tomorrow and push the tasks to beaker.

The wrapper fixed top's test suite.

tcsh results have seen some improvement too, but there are still 3 failures (against 5 without PTY wrapper).

There is one failure in common with RHTS.
And remaining 2 failures are not related to TTY.
So far I was unable to get rid of them: from the research I have done it looks like it could be related to nohup.

Modified task works fine on RHTS as well. Submitted to both RHTS and Beaker.

Done.

https://beaker-stage.app.eng.bos.redhat.com/recipes/2459#task21690

http://rhts.redhat.com/cgi-bin/rhts/jobs.cgi?id=168965