Bug 1737967

Summary: dblatex depends on Python 2
Product: [Fedora] Fedora Reporter: Lumír Balhar <lbalhar>
Component: dblatexAssignee: Michael J Gruber <mjg>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 31CC: alex, bkoz, gwync, jchaloup, mhroncok, mjg, ndbecker2, nforro, pertusus, pviktori
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-10-16 08:47:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1698500, 1738192    

Description Lumír Balhar 2019-08-06 12:09:37 UTC
Python 2.7 will reach end-of-life in January 2020, over 9 years after it was released. This falls within the Fedora 31 lifetime.
Packages that depend on Python 2 are being switched to Python 3 or removed from Fedora: https://fedoraproject.org/wiki/Changes/F31_Mass_Python_2_Package_Removal#Information_on_Remaining_Packages
Python 2 will be retired in Fedora 32: https://fedoraproject.org/wiki/Changes/RetirePython2

To help planning, we'd like to know the plans for dblatex's future. Specifically:


- What is the reason for the Python2 dependency? (Is it software written in Python, or does it just provide Python bindings, or use Python in the build system or test runner?) 

- What are the upstream/community plans/timelines regarding Python 3?

- What is the guidance for porting to Python 3? (Assuming that there is someone who generally knows how to port to Python 3, but doesn't know anything about the particular package, what are the next steps to take?)


This bug is filed semi-automatically, and might not have all the context specific to dblatex.
If you need anything from us, or something is unclear, please mention it here.

Thank you.

Comment 1 Michael J Gruber 2019-08-06 14:29:28 UTC
1) dblatex is written in Python 2, the code is around 19k lines in 100 files.


2) Upstream had its last code commit in 2017, I don't see any plans going forward.


2/3) I might a try a straight 2to3 port but don't see much sense in investing a lot of effort into converting a dead project. (I'm doing this investment for another project already.) In fact, the dependencies are the only reason to keep dblatex alive at all.

3) Since dblatex is about text processing the string type change may make this conversion non-trivial.

Comment 2 Ben Cotton 2019-08-13 16:50:43 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to '31'.

Comment 3 Ben Cotton 2019-08-13 17:38:53 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to 31.

Comment 4 Lumír Balhar 2019-08-14 07:35:25 UTC
The current plan is to remove packages with dependency on Python 2 from Fedora 32 in the middle of November 2019. If you want to keep your package in Fedora after that date and you cannot port it to Python 3 yet, you need to request a FESCo exception for the package and all its Python 2 dependencies (even transitive) [1]. If you don't want to maintain it anymore, please orphan the package.

If you're considering filing the exception request, let us know. We can help (for example, we can help find all the dependencies).

[1] https://fedoraproject.org/wiki/Changes/RetirePython2#FESCo_exceptions

Comment 5 Michael J Gruber 2019-08-14 12:52:59 UTC
While I don't have any personal interest in dblatex, the packagers of 14 packages which build-require it may, or the packagers of 153 packages which build-require asciidoc which build-requires dblatex.

But I ran dnf repoquery on F30, I'm sure everyone on F31 is off py2-based packages by November...

In fact, often dblatex is the "cheap" (simple) way to get PDF from asciidoc via docbook which is not cheap (compute-wise) at all. But typical upstreams change their doc tool chain only reliantly, because all these tool chains have their individual issues which you work around once to be done with it.

But I'll keep you posted on the 2to3 progress.

Comment 6 Michael J Gruber 2019-09-02 20:38:23 UTC
So, while I managed to port a different package to py3, I have to give up on dblatex.

To a large part, dblatex converts back and forth between different encodings when it parses TeX files and its support files. Due to the missing separation between strings and bytes in py2 dblatex can handle "strings with pending decoding" on the same footer as "decoded ascii" etc, which is not possible in py3. I've tried hard to get a clear picture on what are real strings and what bytes in the codebase and got some things to work, but not a fully working dblatex.

So I will let dblatex go its way in F32/rawhide. As I understand, this will be taken care of automatically in November.

Comment 7 Lumír Balhar 2019-09-03 10:57:34 UTC
If you want to orphan it, do it rather sooner than later so somebody interested may take it.

Comment 8 Michael J Gruber 2019-09-10 08:26:37 UTC
Update: Life signs on dblatex, various contributors for a port (more details on fedora-devel)

So for now please no automatic orphaning/killing until we see whether these porting attempts succeed. Should be clearer within a week.

Comment 9 Gwyn Ciesla 2019-09-10 13:57:48 UTC
FYI I've rebundled python-which in dblatex and retired python-which.

Comment 10 Michael J Gruber 2019-09-10 14:07:31 UTC
(In reply to Gwyn Ciesla from comment #9)
> FYI I've rebundled python-which in dblatex and retired python-which.

Have you?

You removed the patch which made dblatex use the external which. You removed the line which removes all bundled libs, which is more than just which.

Now, since we're discussing dblatex's future on fedora-devel and here and on dblatex-devel and are no dblatex maintainer I ask you: Is a pull request too much to ask for before pushing straight to master?

The solution for dlatex-py3 will be different, obviously.

Comment 11 Gwyn Ciesla 2019-09-10 14:23:18 UTC
Apologies. I was under the impression the disable-debian patch kept that piece inactive, and that the dbtexmf contents were unused.

Would you like PR removing the rest but leaving only which?

Comment 12 Michael J Gruber 2019-09-10 14:38:05 UTC
(In reply to Gwyn Ciesla from comment #11)
> Apologies. I was under the impression the disable-debian patch kept that
> piece inactive, and that the dbtexmf contents were unused.
> 
> Would you like PR removing the rest but leaving only which?

No, it's just that I'm somewhat under pressure getting everyone coordinated, the existing patches out in an appropriate form (dblatex just had a new release) and a package out for us to get tested against depending packages. An unexpected push to master is a bit of a trigger ;) 

Which will be replaced differently (shutil.which) so that nothing needs to be bundled.

Comment 13 Miro Hrončok 2019-10-15 10:45:44 UTC
There has been a thread on the devel mailing list:

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/KTBDW4M2K5NY72EFOKEWBE6XF6RCJEUX/#HJN76PLJRRRVMJPOCIV56WYOBFJ2ECRQ

I wonder whether we should do the switch in rawhide now?

Comment 14 Michael J Gruber 2019-10-15 11:22:21 UTC
(In reply to Miro Hrončok from comment #13)
> There has been a thread on the devel mailing list:
> 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/
> thread/KTBDW4M2K5NY72EFOKEWBE6XF6RCJEUX/#HJN76PLJRRRVMJPOCIV56WYOBFJ2ECRQ
> 
> I wonder whether we should do the switch in rawhide now?

I'm fine with using rawhide as a test-bed. That's what it's for.

I've been holding back because discussions on the upstream list were somewhat slow (including mine), the last from a week ago read:

> Sorry too for the delay. Thank you very much for the patches that fix
> most of the things! Yet, I have been struggling with some cases that
> don't work, but I'm confident that I'm close to the end. Encoding,
> decoding, bytes and strings are more painfull with python 3. I guess
> there's a better method to address these features with the new
> language but I'll look for a more consistent solution later...

So while the end appeared to be near it hasn't happened yet. I'll push to rawhide what I have.

Comment 15 Miro Hrončok 2019-10-15 11:30:02 UTC
Thanks. We can always revert if it proves to be really broken.

Comment 16 Michael J Gruber 2019-10-15 11:35:25 UTC
bodhi submitted dblatex-0.3.11-2.fc32 to stable
        https://bodhi.fedoraproject.org/updates/FEDORA-2019-4dd6808ecd