Bug 1047167 - scipy optimize lbfgsb code prints text even if disp==0
Summary: scipy optimize lbfgsb code prints text even if disp==0
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: scipy
Version: 20
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Thomas Spura
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-12-29 18:31 UTC by bob
Modified: 2015-02-21 04:25 UTC (History)
6 users (show)

Fixed In Version: scipy-0.12.1-3.fc20
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-02-21 04:25:22 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
sample screen output with erroneous print to screen (1.34 KB, text/plain)
2014-01-13 01:21 UTC, bob
no flags Details

Description bob 2013-12-29 18:31:48 UTC
Description of problem:

When performing scientific computing/calculations, the scipy package is inappropriately writing commentary text to screen.  Scipy is supposed to suppress screen commentary when disp==0, nonetheless, scipy is writing a lot of garbage text commentary to screen, destroying on-screen content.

This problem was fixed upstream in April.  I reported it as a bug in F18 but it was never fixed.  It remains unfixed in F20.

Version-Release number of selected component (if applicable):

python3-scipy.x86_64

How reproducible:

Always

Steps to Reproduce:
1.  Use scipy to perform scientific computing
2.  Expect suppressed screen output
3.  Ovserve unsuppressed screen output.

Actual results:

Scipy writes inappropriate commentary to screen.

Expected results:

Screen commentary suppressed.

Additional info:

Fixed upstream back in April 2013:

https://github.com/scipy/scipy/issues/2261

Comment 1 Orion Poplawski 2013-12-30 03:18:44 UTC
Ah, so this isn't fixed until 0.13.  So do we update F20 to 0.13.2 or just patch this.  Any issues going from 0.12.1 to 0.13.2?

Comment 2 bob 2014-01-13 01:21:35 UTC
Created attachment 849097 [details]
sample screen output with erroneous print to screen

The attached sample screen output demonstrates the difference between the desired screen output (with successfully suppressed commentary as in previous versions of scipy) and the current screen output, which fails to suppress unwanted commentary written to screen.

Comment 3 bob 2014-04-02 20:48:21 UTC
Just wondering if any decision has been made about updating F20 to 0.13.2, or patching the current version.  It's been three months since I filed the second bug report, and one year since the problem was fixed upstream... but the upstream fixes still aren't being incorporated into Fedora.  

A year seems like an unreasonably long time to wait for a bug that's been fixed upstream to be incorporated into a cutting-edge linux distribution.

Comment 4 bob 2014-10-26 20:17:55 UTC
this bug persists in F20 even though it was fixed upstream in April 2013 -- 18 months ago.  I see that the new post-fix package will be part of F21.  Is there any chance it could be added to the F20 stable repository?

Comment 5 Fedora Update System 2014-10-27 14:36:01 UTC
scipy-0.12.1-2.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/scipy-0.12.1-2.fc20

Comment 6 Fedora Update System 2014-10-28 06:44:02 UTC
Package scipy-0.12.1-2.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing scipy-0.12.1-2.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-13772/scipy-0.12.1-2.fc20
then log in and leave karma (feedback).

Comment 7 bob 2014-10-29 19:04:03 UTC
Works for me.  :)

I left feedback, but it doesn't seem to have increased the karma count.

Comment 8 Orion Poplawski 2014-10-29 19:44:44 UTC
Yeah, you need to be logged in to make karma count.  Thanks for testing though.

Comment 9 bob 2014-11-03 18:59:03 UTC
Found another problem.  For the most part, all of the comments have now been silenced, with the exception of one:

"ascent direction in projection gd = ..."

This results in the interruption of on-screen data presentation.  For example, I have a status bar that is displayed as one of scipy's functions iterates, which produces output that should look like this:

Find Solutions : [****************]

With the inappropriate writing of comments to screen, the output looks like this:


Find Solutions : [  ascent direction in projection gd =    3.4351693296130657E-007
****  ascent direction in projection gd =    3.3637004228124568E-009
  ascent direction in projection gd =    6.3770073492238517E-007
**  ascent direction in projection gd =    2.4667536965736238E-007
  ascent direction in projection gd =    7.5293195786678251E-006
*  ascent direction in projection gd =    2.0071489327006647E-008
***  ascent direction in projection gd =    6.2247249962986455E-008
******

Comment 10 Thomas Spura 2014-11-04 11:58:58 UTC
(In reply to bob from comment #9)
> Found another problem.  For the most part, all of the comments have now been
> silenced, with the exception of one:
> 
> "ascent direction in projection gd = ..."

This is actually a warning or even an error. From the sources:
c the directional derivative >=0.
c Line search is impossible.
write(6,*)' ascent direction in projection gd = ', gd

So one can argue, if this should be printed to stderr instead...
Yet this wouldn't solve your interruption as you would still redefine sys.stderr and write anything to /dev/null instead to the screen.

Comment 11 James 2014-11-05 13:59:10 UTC
I also am interested in this bug.
I have solutions whose convergence in optimize.fmin_l_bfgs_b() is very sensitive to small changes in input data.
I notice the code above is in Fortran, meaning it could be a mature dusty deck that is thoroughly debugged.  If so, why are there now problems with display debug spew, and Comment 10 hints that some could be warnings or errors?

What is exactly the revision history of optimize.fmin_l_bfgs_b(), and why are there problems now?  What functional testing is done to ensure integrity of fmin_l_bfgs_b()?

Thanks.
James

Comment 12 Thomas Spura 2014-11-05 15:24:02 UTC
(In reply to James from comment #11)
> I also am interested in this bug.
> I have solutions whose convergence in optimize.fmin_l_bfgs_b() is very
> sensitive to small changes in input data.
> I notice the code above is in Fortran, meaning it could be a mature dusty
> deck that is thoroughly debugged.  If so, why are there now problems with
> display debug spew, and Comment 10 hints that some could be warnings or
> errors?

The comments in the code hint that line search was not possible.
For implications you could consult the source code or ask the scipy developers what that means.

> What is exactly the revision history of optimize.fmin_l_bfgs_b(), and why
> are there problems now?  What functional testing is done to ensure integrity
> of fmin_l_bfgs_b()?

You can find the tests at [1] and full history of lbfgs at [2].
It seems, in a later commit [3], the scipy developers remove this print too, so I'll push a new update soonish.

[1] https://github.com/scipy/scipy/tree/master/scipy/optimize/tests
[2] https://github.com/scipy/scipy/commits/master/scipy/optimize/lbfgsb/
[3] https://github.com/scipy/scipy/commit/ea96f8697deea203675c44b3d24e427e552c4de7

Comment 13 Fedora Update System 2014-11-07 02:34:51 UTC
Package scipy-0.12.1-3.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing scipy-0.12.1-3.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-13772/scipy-0.12.1-3.fc20
then log in and leave karma (feedback).

Comment 14 Fedora Update System 2015-02-21 04:25:22 UTC
scipy-0.12.1-3.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.