I don't think it attempts an non-SSL connection actually, the message comes from spice-channel.c:

reconnect:
    c->sock = spice_session_channel_open_host(c->session, channel, c->tls);
    if (c->sock == NULL) {
        if (!c->tls) {
            CHANNEL_DEBUG(channel, "connection failed, trying with TLS port");
            c->tls = true; /* FIXME: does that really work with provided fd */
            goto reconnect;
        } else {
            CHANNEL_DEBUG(channel, "Connect error");
            emit_main_context(channel, SPICE_CHANNEL_EVENT, SPICE_CHANNEL_ERROR_CONNECT);
            goto cleanup;
        }
    }

The actual connection is done in spice_session_channel_open_host from spice-session.c which starts with

    if ((use_tls && !s->tls_port) || (!use_tls && !s->port))
        return NULL;

c->tls and use_tls are FALSE or "connection failed, trying with TLS port" wouldn't appear, and s->port is null as well, so NULL is returned before any connection is attempted.

sent to ML

   channel: improve debugging message
    
    The open_host() can return FALSE when the connection is discarded or
    skipped. Improve the message to not indicate a failure.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=858232

in spice-gtk-0.14-5.el6

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-0343.html