Bug 1462838

Seems to work with for example ftp://mirrors.ocf.berkeley.edu/centos/7.3.1611/os/x86_64/ . Does that URL work for you?

There is a newer virt-manager release. Please try git code:

git clone git://github.com/virt-manager/virt-manager
cd virt-manager
./virt-install ...

I have not yet tried the newer virt-manager you mentioned but I will do so and report back.  

However, using the same version as previously, the URL ftp://mirrors.ocf.berkeley.edu/centos/7.3.1611/os/x86_64 does work.

The ftp site that is not working is a local mirror so it would appear that the issue is there.  I cannot see any difference in the files being served by FTP between the two sites.  The local mirror also works fine for yum updates and delivering the kickstart file.  

Specifically the local ftp server has .treeinfo and it is identical to .treeinfo on the Berkeley FTP server.  

Is there a more verbose debug level that can be enabled in virt-install?

Thank you,
Kevin

No, there isn't any higher debug level than --debug. You can try poking in the code in virtinst/urlfetcher.py, but better to do it from the git directory. Maybe it's some configuration setting of the FTP server

I traced the issue to the function _FTPURLFetcher._grabber()

When attempting to get the size of the .treeinfo file from the FTP server (proftpd) the server is returning a "550 SIZE not allowed in ASCII mode."

As the lower function is wrapped by _URLFetcher._grabURL and inside a try I would have expected that the exception be raised saying the file could not be acquired but that is not the behavior I'm seeing.  It appears that the call

"size = self._ftp.size(urlparse.urlparse(url)[2])"

does not set the value of size and does not raise any exception that is caught.  

Doing some googling on the FTP error and there are good reasons why SIZE should not be allowed on ASCII files.  However the ftpd configuration is valid and virt-install should probably be detecting this failure mode and giving a more accurate error since the file is available.

From RFC3659 (Extensions to FTP) Section 4.2 "The presence of the 550 error response to a SIZE command MUST NOT be taken by the client as an indication that the file cannot be transferred in the current MODE and TYPE. A server may generate this error for other reasons -- for instance if the processing overhead is considered too great."

A possibly simple workaround is for virt-install to request .treeinfo as a binary file assuming that the different line end conventions of the files will not prevent .treeinfo from being processed.  Since the proftpd is local to my environment I was able to changed the "DefaultTransferMode" to binary and virt-install is working as expected.  It doesn't appear the line end conventions are a problem.

I appreciate your assistance with this issue.

Kevin

Cool thanks for digging into this. So your server is defaulting to ascii mode which explains it. Does this patch fix things for you?

iff --git a/virtinst/urlfetcher.py b/virtinst/urlfetcher.py
index 5e10bb52..152b076a 100644
--- a/virtinst/urlfetcher.py
+++ b/virtinst/urlfetcher.py
@@ -224,6 +224,8 @@ class _FTPURLFetcher(_URLFetcher):
             self._ftp = ftplib.FTP()
             self._ftp.connect(parsed.hostname, parsed.port)
             self._ftp.login()
+            # Force binary mode
+            self._ftp.voidcmd("TYPE I")
         except Exception as e:
             raise ValueError(_("Opening URL %s failed: %s.") %
                               (self.location, str(e)))

Kevin does that patch fix things for you? I'd like confirmation before I push it to virt-manager git

I was finally able to test this and it works as advertised.

Very sorry for the long delay in responding and thank you so much for the help.

Kevin

Thanks for confirming, that change is pushed upstream now

commit e902fa5550cc09538429b0fa828fb31aa2706d01 (HEAD -> master)
Author: Cole Robinson <crobinso>
Date:   Thu Aug 17 16:10:46 2017 -0400

    urlfetcher: force binary mode with FTP servers (bz #1462838)