Bug 969905

Thanks for a bug report. I have my Exchange server also hidden behind vpn, and I'm also using vpnc to connect to the network, but I do not use NetworkManager for that, I run the vpnc on my own from a terminal instead, which might be the only difference between our setup, I guess? I basically run vpnc, and only then I run Evolution, and everything works as expected. I see you've also installed evolution-ews, thus what about trying it, instead of evolution-mapi? evolution-ews is better in particular things, especially when your Exchange server is 2007 or newer (the newer the better).

With evoluiton-mapi I would try this:
a) close evolution
b) connect to vpn
c) close other running evolution processes (ps ax | grep evolution), while you
   close/kill evolution-source-registry before evolution-calendar-factory,
   because the newly run calendar factory can run source registry
d) run evolution
e) go to Edit->Preferences->Mail Accounts-><MAPI account>->Edit, and check
   whether the server address is filled properly and then click
   the Authenticate button, to re-setup the account internal configuration

I guess it'll help. Please let me know if it did.

Thanks for your answer. 

I should properly have written, that I have tried vpnc from console and that I got no different effect from that (sorry).

I have never been able to get contacts to work with evolution-ews. The reason I have evolution-ews installed is because it was default. I am also planning to use a openchange server on another location and figure that ews is not working with what. I was not able to get any success out of the a) to e) steps I have also tried that before.

I have made a new wireshark sniff and found the difference in vpn connection and then it is not. It seems that the DCERPC protocol is missing then i am connected to vpn. I only see DNS requsts/awnser then I am connected to vpn.

/Stig

I think you get the "never ending search", because evolution-source-registry is stuck with other password prompt, which was initiated before the vpn was connected. I'm currently using Fedora 19, and it works fine here.

You are right that OpenChange server doesn't support EWS protocol, at least for now. I heard the OpenChange developer that they plan EWS support, in some future.

Anyway, could you install debuginfo packages for evolution, evolution-data-server, evolution-mapi and openchange (make sure the version numbers of packages match their binary package versions), then get evolution to this neverending state of refreshing folder, and get a backtrace of running evolution and evolution-source-registry processes. You can get the backtrace with command like this:
   $ gdb --batch --ex "t a a bt" -pid=`pidof evolution` &>bt.txt
for evolution.

Please make sure you'll not expose any private information in backtraces, because it can be waiting in authentication routines, thus possibly exposing your password in a clear form. I usually search at least for "pass" (quotes for clarity only), to search for anything similar to "password".

I found a clean machine to install Fedora 19 on and used the default installation with gnome-desktop as I always do. Installed wireshark for sniffing and evolution-mapi - configurede my vpnc (terminal version and networkmanager too).

connected to the remote-site (terminal vpnc) and killed the evolution-source-registry process and the other evolution processes and started evolution. Going throw the new-account procedure and selected mapi, typed the servers FQDN and my username and domain-name - selected secure-connection and tried to authenticate - then I have typed my password Evolution says "host not found" - I can see in my wireshark sniff, that it is asking for the server and the DNS is awnswering back with the right IP address. I can open er terminal and ping/scan the server so it is reachable.

If I do the same stuff in a virtualbox installation of the same Fedora 19 installation - use the hosting system to estaplish vpnc to the remote-site. Then I am able to do all the stuff as if I was local on the remote-site LAN.

Can I ask, what kind of VPN are you using at your remote-site ? Even tough I have tried two different Cisco VPN units. What desktop enviroment are u using.

I will do a debuginfo install of evolution, then I have made my new account again - on the real machine and not the virtualbox (to slow for debugging).

/Stig

(In reply to stig from comment #4)
> then I have typed my password Evolution says "host not found"

I guess the server redirected to a different machine. It did it for me too, in the past, thus I had an entry in my /etc/hosts to bind the returned address name to the same server.

On your Fedora 19, please run evolution like this:
   $ LIBMAPI_DEBUG=15 evolution
and then repeate the Authnetication. You should see detailed information with conversation between evolution-mapi and your Exchange server, which should, I hope, shed a bit of light on the issue.

> Can I ask, what kind of VPN are you using at your remote-site ?

To be honest, I do not know, I do not maintain the server side at all.

okay! More test done.

I installed fedora 17 64 bit in a virtualbox:

evolution.x86_64                3.4.4-2.fc17
evolution-NetworkManager.x86_64 3.4.4-2.fc17
evolution-data-server.x86_64    3.4.4-5.fc17
evolution-help.noarch           3.4.4-2.fc17
evolution-mapi.x86_64           3.4.4-2.fc17

And this is working perfectly over the vpn tunnel:
- no problem with creating the mapi account
- no problem to syncronize the mail

 I also see the DCERPC protocol talking, just right after DNS request/repley, in wirshark as expected - then sniffing the tun0 interface.

- And "LIBMAPI_DEBUG=15 evolution &>log.txt" did show all the good stuff.

I did the same on fedora 18 64 bit on real hardware, but that should not actual matter anyway.

evolution.x86_64                             3.6.4-3.fc18
evolution-data-server.x86_64                 3.6.4-4.fc18
evolution-ews.x86_64                         3.6.4-1.fc18
evolution-help.noarch                        3.6.4-3.fc18
evolution-mapi.x86_64                        3.6.4-1.fc18

And here is what happend, then the vpn tunnel is activated
- unable to create the mapi account ( workaround was to be local on LAN at the remote site ).
- unable to syncronize the mailfolders

I do not see the DCEPRC protocol talking at all on any network interfaces I had sniffed with wireshark.

- And "LIBMAPI_DEBUG=15 evolution &>log.txt" only shows that the debug settings is 15 for them all.

Fedora 19 is behaving as Fedora 18

so - fedora 18 and 19 is not doing any DCERPC babbling on the wire - then VPN is activated (As I mentioned before I only see the DNS request and the reply for the server - so this is working).

/Stig

I rechecked, on Fedora 19, just to be sure, and it works for me. Is it possible your /etc/resolv.conf was not updated for some reason? Mine looks like:
   $ cat /etc/resolv.conf
   #@VPNC_GENERATED@ -- this file is generated by vpnc
   # and will be overwritten by vpnc
   # as long as the above mark is intact
   # Generated by NetworkManager
   nameserver 123.123.123.1
   nameserver 123.123.123.2
   search server.com

Could you also check /etc/hosts, just in case? It should not mention the server you try to connect to. Note the NetworkManager notice in resolv.conf, it's because the network interface is handled by NetworkManager (can the problem lie somewhere there?).

I've really not much idea what can be wrong here, but it seems to be lower than in evolution-mapi.

I have not made changes to the /etc/hosts file and the /etc/resolve.conf is generated by NetworkManager (total new installation). The /etc/hosts file is clean and total basic - and the /etc/resolv.conf is holding 3 nameservers,

the 2 first is the remote site DNS and the last is a local DNS.

And it does not matter that I use vpnc terminal version or networkmanager. 

I still do not think, that it is the issue here - because as mention before - the DNS works (have tried to use pure IP address in the server field in the mapi account profile). DCERPC traffic i guess is part of the evolution-mapi and that part is not showing up in the network traffic sniff off tun0 or any other interfaces.

- And I use vpn often and have no other issues with vpn for any of the other daily tasks - even the other imap connections changes to the right server.

/Stig

OK. Evolution-mapi uses OpenChange's libmapi, which uses samba for DCERPC, thus the reason might be somewhere in samba (any ports blocked by firewall?). Please install openchange-client, then try to configure a profile with mapiprofile, with command like this:
   $ mapiprofile -c -P testProfile -I exchange.com -D domain -u user -p pass
which should fail the same as evolution-mapi fails.

(In reply to Milan Crha from comment #9)
> Please install openchange-client, then try to configure a
> profile with mapiprofile, with command like this:
>    $ mapiprofile -c -P testProfile -I exchange.com -D domain -u user -p pass
> which should fail the same as evolution-mapi fails.

I have tried to disable firwalld and done a setenforce 0 - without any results.
And it still work then I use a virtualbox and the hosting part is using vpn - but not then I tried using vpn directly on the host that I use the Evolution - so I do not think has anything to do with it.

The outcome of the mapiprofile is:

Profile testProfile completed and added to database /root/.openchange/profiles.ldb


So I guess from the wireshark sniff, that this part is working.

/Stig

Hmm, ok, could you run it as a regular user, please? Could you remove ~/.local/share/evolution/mapi-profiles.ldb and then run evolution from a console, go to Edit->preferences->Mail Accounts-><mapi account>->Edit->Receiving Email->and click Authenticate button there, please? It will re-setup MAPI account from scratch.

yep! sorry my bad - but it also work with a plain user.

I removed the mapi-profiles.ldb and tried the Authenticate button - It is saying "Searching for server - please wait" and eventually says "host unreacheable"

/Stig

Hmm, I'm getting out of idea.

If the mapiprofile works, then also evolution should work, unless you've set any restriction on it, either on firewall (we checked that above, result was negative), or maybe proxy, either system or evolutions (Edit->Preferences->Network Preferences). That the only evolution is not using the vpn connection is quite suspicious, and looks like a (system) setup issue, from my point of view. As I mentioned above, I use pretty the same setup as you, and it works for me (I do not use IPv6, I use only IPv4). I would recheck /etc/hosts and other similar settings, maybe they are stale from your update - aka they worked fine in F17, but once you updated to F18 they make trouble.

I know it sound a bit odd - but the F17 was a clean and fresh installation nothing done to it except the installation of wireshark, evolution-mapi and configuration of VPN.

F18 was not upgraded from F17 et is also a clean and fresh installation
F19 was not upgraded from any of the others - stil clean installation.

The above is just to state that it is different machines I am trying on (but I does not matter if it is physical or virtual machines).

I do not see anything wrong with evolution then using imapx or even the evolution-ews or any other application over VPN. This of cause only happens with F18 and F19 :D

If I on my F18 do install a F19 in a virtual machine and connect to the VPN with my F18 then the F19 have no problem to connect to the exchange server. So I don't think there is any firewall issue and I tried nmap scanning the exchange server and there is nothing blocked for the VPN connection - same ports is open.

If it was some thing with the hosts file - I guess that it will have worked if then I used the ip address only. I have also disable the firewalld on the client. I guess that if it have something to do with firewalld or the hosts file - I could not have done the above command $ mapiprofile -c -P testProfile -I exchange.com -D domain -u user -p pass

with success. Other thing i could state is as network technician, that I never use the hosts file to do anything, because it make more troubles than it solves (DNS is the way or directly use of IP address:-D ).

I have checked the resolve file and I have even explicit added one DNS server I know is on the remote site.


I also only use ipv4 - not that I think ipv6 is doing anything at all.

The route-table is also correct - nothing fancy here either.

- So there must bee something that does not goes right in the evolution-mapi against libmapi/SAMBA -> seems that evolution-mapi is never getting the server information back from libmap/SAMBA even that I can see the DNS is resolving the request server and is replying with the IP address and that is maybe why then the ip address is applied instead of the FQDN is notworking either.

/Stig

I asked about IPv6, because I saw a bug in libsoup (rather 'exhibited by libsoup', than 'in libsoup' itself), where was properly recognized IP address of a FQDN, but instead of using IPv4 interface, the dead end of IPv6 was used, resulting in timeout. It seems like the same thing you face. Why it works fine from a virutal machine, but not from a real machine I do not know, this is out of my knowledge. In any case, I would make sure that you've disabled the IPv6 interface on your network card. Then install debuginfo packages for all samba packages you've installed, same as for openchange packages, then run evolution as this:
   $ LIBMAPI_DEBUG=15 evolution
and try to 'Authneticate' the account. When it'll be stuck waiting for the server response, get a backtrace:
   $ gdb --batch --ex "t a a bt full" -pid=`pidof evolution` &>bt.txt
The backtrace can show your user name, password, server address and other sensitive data in a clear form, thus make sure you'll not upload it here with them. I usually search for "pass" (quotes for clarity only), at least.

The backtrace will provide us an information where the samba code is wating.

Created attachment 759465 [details]
Backtrace for evolution

openchange.x86_64                             2.0-1.fc19
openchange-client.x86_64                      2.0-1.fc19
openchange-debuginfo.x86_64                   2.0-1.fc19

evolution.x86_64                              3.8.3-1.fc19
evolution-data-server.x86_64                  3.8.3-1.fc19
evolution-data-server-debuginfo.x86_64        3.8.3-1.fc19
evolution-debuginfo.x86_64                    3.8.3-1.fc19
evolution-help.noarch                         3.8.3-1.fc19
evolution-mapi.x86_64                         3.8.3-1.fc19
evolution-mapi-debuginfo.x86_64               3.8.3-1.fc19

samba-client.x86_64                       2:4.0.6-3.fc19
samba-common.x86_64                       2:4.0.6-3.fc19
samba-debuginfo.x86_64                    2:4.0.6-3.fc19
samba-libs.x86_64                         2:4.0.6-3.fc19

I have disabled the IPv6 protocol, but that does not change anything. I only told you, that it worked from the virtual machine then it is connecting throw the hosted machines VPN to state that it is not a firewall issue.

The debug does not show anything odd $ LIBMAPI_DEBUG=15 evolution

(evolution:2305): Gtk-CRITICAL **: gtk_entry_set_text: assertion `text != NULL' failed

(evolution:2305): Gtk-CRITICAL **: gtk_entry_set_text: assertion `text != NULL' failed

(evolution:2305): Gtk-CRITICAL **: gtk_entry_set_text: assertion `text != NULL' failed

(evolution:2305): Gtk-CRITICAL **: gtk_entry_set_text: assertion `text != NULL' failed

(evolution:2305): Gtk-CRITICAL **: gtk_entry_set_text: assertion `text != NULL' failed
INFO: Current debug levels:
  all: 15
  tdb: 15
  printdrivers: 15
  lanman: 15
  smb: 15
  rpc_parse: 15
  rpc_srv: 15
  rpc_cli: 15
  passdb: 15
  sam: 15
  auth: 15
  winbind: 15
  vfs: 15
  idmap: 15
  quota: 15
  acls: 15
  locking: 15
  msdfs: 15
  dmapi: 15
  registry: 15

remember that it is not only then using DNS the problem occurs - it does not work with pure IP addressing also. And the FQDN is resolved - but still no luck.

Could there be a difference in the 64 bit version and 32 ?

Sorry for posting the Backtrace before this post - that was not suppose to happend.

/Stig

Okay! It does not matter that it is 32 or 64 bit - samme issue.

/Stig

Hmm, the backtrace shows that the evolution is idle, it doesn't do anything. I expected it being stuck in a network call or anything.

I have made a backtrace of the other situation, where the account is created and working on the LAN at the remoteside. Then I got back and started the evolution and it hangs at "Connecting to 'Exchange MAPI server (my FQDN servername)'. I hope that situation could sheet some light at the problem.

backtrace would be in the next post.

/Stig

Created attachment 761082 [details]
backtrace of evolution

This is a bactrace of Evolution  then account is created in advance at the remote LAN.

Thanks for the update. The backtrace shows possible deadlock, which might be the reason why the LIBMAPI_DEBUG doesn't show anything useful. I'll check the code, but it seems we are getting somewhere now.

Created attachment 761129 [details]
proposed ema patch

for evolution-mapi;

This should avoid the deadlock shown in the last backtrace. I built a test package for you [1], it's for Fedora 18 (same as this bug report). Please download package for your architecture and install it. Then restart evolution and retest.

Here [2] is currently building test package for Fedora 19, with the same change.

It'll be good to install also debuginfo package for evolution-mapi, thus the backtrace will contain more detailed information. (Before attaching it here, make sure it'll not expose any private information, like passwords, user names, email or server addresses and such. The previous backtrace didn't expose any only because you didn't have installed debuginfo package.)

[1] http://koji.fedoraproject.org/koji/taskinfo?taskID=5501937
[2] http://koji.fedoraproject.org/koji/taskinfo?taskID=5501954

It had certainly made a change - now I got a "Failed to open folder. The reported error was 'Folder list is not available in offline mode'." every time.

I will try to make a backtrace with mapi-debuginfo package also - just lost the track of what machine I had installed the package on.

It did not change anything for the "creating new account" situation - it still belive that the host is not reachable.

/Stig

Nice, that's a good step forward. I'd say that the
  $ LIBMAPI_DEBUG=15 evolution
should give more output than is shown in comment #17. Could you try that too, please?

I moved the deadlock part upstream, as [1], together with the patch. Let's continue with the investigation here.

[1] https://bugzilla.gnome.org/show_bug.cgi?id=702276

? Then you write "continue with the investigation here" do you mean here at bugzilla.redhat.com or do you want me to create a account on bugzilla.gnome.org ?

- But as you requested in comment 25 ( hope it is okay that I removed my name and my company mail).

The first run was mostly like this:

(evolution:2607): evolution-mail-WARNING **: Failed to refresh folder 'my.email.dk: Postkasse - My Name/Deleted Items': Host unreachable


(evolution:2607): IBUS-WARNING **: Unable to connect to ibus: Could not connect: Connection refused

the rest is also  evolution-mail-WARNING but different folders.

I did not get the red banner with "Failed to open folder. The reported error was 'Folder list is not available in offline mode'." the first time I was running the evolution.

Second time I got the red banner again:

I got all the same (evolution:3105): evolution-mail-WARNINGS about failed to refresh folders.

(evolution:3105): IBUS-WARNING **: Unable to connect to ibus: Could not connect: Connection refused **

(evolution:3105): WARNING **: Folder list is not available in offline mode


/Stig

I meant 'here', with 'here', thus Red Hat bugzilla. :)

The LIBMAPI_DEBUG should show more than that, it seems like it didn't work as expected. Could you do it differently, please:
   $ export MAPI_DEBUG=15
   $ export LIBMAPI_DEBUG=15
   $ evolution

Created attachment 762179 [details]
extended debug export evolution run

I will narrow down my test bench to only containing a fedora 19 beta in a virtualbox with the evolution-mapi and evolution-mapi-debuginfo you created in comment 23. I think we shall only focus on the settings with the setup where I got a account created in advance.

I don't see anything different from the other run - it is still complaining about host unreachable.

I tried with out the VPN connection and the I got host unresolved or unknown instead.

/Stig

I will still have a version of F19 beta on hardware to exclude any virtualbox errors - and they will all be 64 bit versions.

/Stig

Oh, I think I know from where the "Host unreachable" comes, it might be GNetworkMonitor, which tries to reach your server at port 135. If it fails, then it claims this 'Host unreachable' error. That may also explain why there is no samba debug output, it doesn't get that far.

Could you run evolution under gdb like below, please?
   $ gdb evolution --ex "b can_reach_mapi_server" --ex r \
       --ex "p server_address" --ex "f 1" --ex "p *profile" --ex "d b 1" --ex c
which will show what server address is used for this test. It is stored in the profile, thus I'd even suggest to re-try Authentication, with all the debugging from comment #28 on.

I guess that either there's a wrong address stored in the profile, or the server rejects port 135 for some reason (it can even redirect?). There is currently no way to disable this test from outside, it's used to check server availability (which is the reason why one of the errors mentions "offline"). The logging during Authentication shows server name when connecting to port 135, please check if it differs or not (basically export the debugging variables from comment #28 and run evolution like: "$ evolution &>log.txt" and when the authentication is done then close evolution and search resulting log.txt file for 135, or your server address).

Breakpoint 1, can_reach_mapi_server shows the correct "servername.mydomain.tld"
also profname shows my correct "username@domain.tld" and password is correct also.

The reauthentication do not show anything - only that evolution thinks it is offline. There is no servername or port 135 - I tested the port 135 at the server with telnet in a terminal and godt a connection

telnet servername.mydomain.tld 135
Trying (the right server ip)...
Connected to servername.mydomain.tld.
Escape character is '^]'.

Thies lines is the only thing that differs.

(evolution:2253): WARNING **: Folder list is not available in offline mode

volution:2253): Gtk-CRITICAL **: gtk_entry_set_text: assertion `text != NULL' failed

do not know why this line is cut off in the front.

I also notis that - I cant do any offline/online connect on the icon in the left lower side.

/Stig

Created attachment 762419 [details]
test.c

(In reply to stig from comment #32)
> I also notis that - I cant do any offline/online connect on the icon in the
> left lower side.

Hmm, OK, then let's start here, because it might be part of the issue. The icon also talks to GNetworkMonitor, and it's disabled when there is no network available. It depends on the GNetworkMonitor backend how it recognizes network availability; usual cause with NetworkManager backend was that the network interface was not managed by NetworkManager.

Please download this test.c, change the "exchange.server.com" with your actual server address, install glib2-devel package and compile it with the command from the first line comment of the test.c file. It tries to access the server on port 135 and also prints whether the network as such is available. It should make testing easier. Also note that both lines it prints should end with "yes", and once they will, also evolution and evolution-mapi will work.

That returns:

test.c:14:2: warning: ‘g_type_init’ is deprecated (declared at /usr/include/glib-2.0/gobject/gtype.h:669) [-Wdeprecated-declarations]
  g_type_init ();

Network available: no
Server 'servername.mydomain.tld' reachable: no
   Error: Host unreachable

I will try to make a wireshark sniff on the tun0 interface to see if there is comming any traffic down the interface.

/Stig

The sniff shows, then I run that program, that the DNS name is resolved and the DNS server is replying with the right ip address.

/Stig

(In reply to stig from comment #34)
> Network available: no

The above is the main problem, the GNetworkMonitor thinks you are not connected to the Internet. Did you make sure that the network interface (like eth0) is managed by the NetworkManager? Or maybe there's missing something installed. Unfortunately, I do not know from where the GNetworkMonitor gets its backend to be used. I'll ask around.

I do not know where in the chain the GNetworkMonitor is - but I have not done anything to the NetworkManager like uncheck the Manages by NetworkManager ( I can't even find that feature any more in F19beta). But is evolution-mapi the only one to use GNetworkMonitor? If not - then I do not understand why the other protocols like imap and ows is able to connect trough the VPN.

Would anything else not be affected by the GNetworMonitor state if it was beliving that it could not reache the internet or is GNetworkMonitor only part of Evolution?

/Stig

evolution itself relies on GNetworkMonitor with the online/offline button, the left-bottom icon you mentioned above. Other evolution providers/backends also depend on GNetworkMonitor accuracy, evolution-mapi as one of them. Evolution-mapi uses GNetworkMonitor to check whether the server is reachable to not let samba do it, because samba timeouts are significantly larger than those used by GNetworkMonitor. If evolution/evolution-mapi are the main users of GNetworkMonitor I cannot tell, I only see that my instance of NetworkManager works fine in Fedora 19 and reports network reachable.

I'm moving this to glib2, because its internals are out of my knowledge.

The last update to

glib2.x86_64 2.36.3-2.fc19

did not do anything different for this bug - the little test.c program in comment 33 do still come out with double negative (no no).

/Stig

The last update to

glib2.x86_64 0:2.36.3-3.fc19

did fix the problem - it now works with evolution and the little test.c program is reporting (yes yes).

/Stig