Bug 1463160 - User folders are now created in /var/spool/imap instead of /var/spool/d/user/dhill/
Summary: User folders are now created in /var/spool/imap instead of /var/spool/d/user/...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: cyrus-imapd
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
Assignee: Jason Tibbitts
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-06-20 09:14 UTC by David Hill
Modified: 2017-06-27 21:35 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-06-27 21:35:57 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description David Hill 2017-06-20 09:14:33 UTC
Description of problem:
User folders are now created in /var/spool/imap instead of /var/spool/d/user/dhill/ after updating cyrus-imapd from 2.5 to 3.0.1

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. Update from 2.5 to 3.0.1
2.
3.

Actual results:
User lost his folders 

Expected results:
Shouldn't happen or some kind of warning should be displayed.

Additional info:
Messages are no longer displayd.

Comment 1 David Hill 2017-06-20 09:50:20 UTC
I don't know if it's related, but if I use cyradm "lm" , I no longer see my user mailbox...

Comment 2 David Hill 2017-06-20 10:09:12 UTC
Going back to 2.5.0 solves the issue.   I've also made sure to follow the upgrade [1] procedure to 3.0.1 but no dice, once upgrade, everything blows up.

[1] https://cyrusimap.org/imap/download/upgrade.html

Comment 3 Jason Tibbitts 2017-06-20 15:36:50 UTC
You're saying that things were in /var/spool/(first letter of user name)/user/(username) before?  That is certainly not remotely correct.

/var/spool/imap is certainly the proper place for these things unless you configure cyrus otherwise.  It's tough to say without knowing what's in your imapd.conf.

If you add something like:

partition-default: /var/spool

to imapd.conf, do things improve?  How about if you don't add that line but move the d directory to be under /var/spool/imap?  You might have to run reconstruct in both cases but maybe not.

Comment 4 David Hill 2017-06-22 20:21:59 UTC
This is what I have:

altnamespace: yes
partition-default: /var/spool/imap
admins: cyrus dhill
sievedir: /var/lib/imap/sieve
sendmail: /usr/sbin/sendmail
hashimapspool: true
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN LOGIN
sasl_auxprop_plugin: sasldb
sasl_auto_transition: no
allowplaintext: no
defaultdomain: zappa.quebec
tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt
# uncomment this if you're operating in a DSCP environment (RFC-4594)
# qosmarking: af13

Comment 5 Jason Tibbitts 2017-06-22 20:35:06 UTC
I don't understand; is that what you have under 2.5 and the partition-default is being ignored?  Or is that what you have under 3.0, in which case it's putting the files exactly where it's supposed to be putting them.

In any case, there was never a Fedora release with 2.5.10 in it; F25 has 2.4.something and F26 will have 3.0.2.  As far as I can tell from what you're saying, the interim builds of 2.5.10 had some bug which caused the mail to go somewhere it really shouldn't be, but that things are now correct.

But if that's the case then this would be something of a "reverse bug report".  So maybe I'm badly misunderstanding, but everything I can see seems to indicate that there was previously a bug but that there is not currently a bug (in this area, at least).

Also, it is quite a bad idea to make a regular user an admin; IDs listed in "admins" should never receive mail.  The imapd.conf manpage should explain this.

Comment 6 Zdenek Dohnal 2017-06-23 07:32:20 UTC
Jason,

what I understood from David's comment #2, he followed upgrade instructions on cyrus website, where is written to install from tarball. David, did I understand it correctly?

Comment 7 David Hill 2017-06-25 14:36:06 UTC
Hello, 

   I have the same configuration for both of them but i adapted my configuration for 3.0.  In 2.5, everything's working alright but as soon as i update to 3.0.1-7, I lose all my folders in my user inbox.   Did I do something wrong?   It's like if it'd be creating shared folders as instead of creating folder in /var/spool/imap/d/dhill/blah, it creates it in /var/spool/imap/b/blah ...

Dave

Comment 8 David Hill 2017-06-25 14:46:00 UTC
I've just retried to upgrade and it has the same behavior again.

That is my current configuration for 3.0.1-7:

configdirectory: /var/lib/imap
altnamespace: yes
partition-default: /var/spool/imap
admins: cyrus dhill
sievedir: /var/lib/imap/sieve
sendmail: /usr/sbin/sendmail
hashimapspool: true
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN LOGIN
sasl_auxprop_plugin: sasldb
sasl_auto_transition: no
allowplaintext: no
defaultdomain: zappa.quebec
tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt
# uncomment this if you're operating in a DSCP environment (RFC-4594)
# qosmarking: af13

Comment 9 David Hill 2017-06-27 15:21:48 UTC
I seem to have fixed it by adding "unixhierarchysep: 0" ... is there a way to add a postconfig RPM warning that warns people about that if it's missing?

Comment 10 Jason Tibbitts 2017-06-27 16:31:41 UTC
I read on the cyrus list that your mail spool is actually located under /var/spool/imap, not under /var/spool as you indicated in the initial report here.  Which might explain why my responses are so confused.

In any case, output during %post scriptlets isn't permitted in Fedora anyway, so that wouldn't really work.  We can hook in a script run before the service starts, but I'm not entirely sure that there's a reasonable way to detect this situation.  If you have a concrete suggestion that would work across the wide variety of different configurations someone might have set up, I'm happy to look into it.

I will amend my release notes request to mention this explicitly, though.

Comment 11 David Hill 2017-06-27 16:41:29 UTC
I though it was fixed but I still have the problem... Is there anything I need to change in my IMAP client in order to make it work ?  Like "Personal Namespace" ?  In thunderbird, it defaults to INBOX/ ...

Comment 12 David Hill 2017-06-27 16:45:42 UTC
Yeah sorry(In reply to Jason Tibbitts from comment #10)
> I read on the cyrus list that your mail spool is actually located under
> /var/spool/imap, not under /var/spool as you indicated in the initial report
> here.  Which might explain why my responses are so confused.
> 
> In any case, output during %post scriptlets isn't permitted in Fedora
> anyway, so that wouldn't really work.  We can hook in a script run before
> the service starts, but I'm not entirely sure that there's a reasonable way
> to detect this situation.  If you have a concrete suggestion that would work
> across the wide variety of different configurations someone might have set
> up, I'm happy to look into it.
> 
> I will amend my release notes request to mention this explicitly, though.

It was a typo it still creating it in /var/spool/imap but it's like if my user was leaking out of its namespace.

Comment 13 Jason Tibbitts 2017-06-27 19:38:01 UTC
Honestly I'm not familiar with all of the ways that cyrus can interact with clients.  If you adjusted unixhierarchysep and altnamespace as documented in https://cyrusimap.org/imap/download/upgrade.html#copy-config-files-and-update and followed the rest of those instructions for running reconstruct and such, then I'm out of ideas.

Since you've already mailed the upstream mailing list, I'll see if there's something useful pointed out there but I just don't think this is an issue in the package itself.

Comment 14 David Hill 2017-06-27 20:25:28 UTC
The more I look and the more it seems like when I upgrade, it simply stops finding the user mailbox.  I did update the mailboxes but it's like if /var/spool/imap/d/user/dhill would be ignored.   Does this make sense?   The folders have cyrus:mail ownership with 700 ... does this behavior changes with 3.0 ?

Comment 15 David Hill 2017-06-27 20:43:29 UTC
IMAP Password: 
              localhost> lm
Archive (\Subscribed \HasNoChildren)                                  
Archives (\Subscribed \HasNoChildren)                                 
DELETED.INBOX.Trash.INBOX^Deleted Messages.59529F68 (\HasNoChildren)  
DELETED.INBOX.Trash.asdf.59529F34 (\HasNoChildren)                    
DELETED.INBOX.Trash.dhill.59529F37 (\HasNoChildren)                   
Junk (\Subscribed \HasNoChildren)                                     
Notes (\Subscribed \HasNoChildren)                                    
Sent (\Subscribed \HasNoChildren)

Comment 16 David Hill 2017-06-27 21:01:35 UTC
This is getting weirder as the following information shows:

As the admin user, we can see all users mailboxes:

localhost> lm
Archive (\HasNoChildren)                                                   
Archives (\HasNoChildren)                                                  
DELETED.user.dhill.Trash.INBOX^Deleted Messages.59529F68 (\HasNoChildren)  
DELETED.user.dhill.Trash.asdf.59529F34 (\HasNoChildren)                    
DELETED.user.dhill.Trash.dhill.59529F37 (\HasNoChildren)                   
Junk (\HasNoChildren)                                                      
Notes (\HasNoChildren)                                                     
Sent (\HasNoChildren)                                                      
user.dhill (\HasChildren)                                                  
user.dhill.Archive (\HasNoChildren)                                        
user.dhill.Archives (\HasChildren)                                         
user.dhill.Archives.2017 (\HasNoChildren)                                  
user.dhill.Cron (\HasNoChildren)                                           
user.dhill.Deleted Messages (\HasNoChildren)                               
user.dhill.Drafts (\HasNoChildren)                                         
user.dhill.Jobboom (\HasNoChildren)                                        
user.dhill.Junk (\HasNoChildren)                                           
user.dhill.Junk E-mail (\HasNoChildren)                                    
user.dhill.Notes (\HasNoChildren)                                          
user.dhill.Read (\HasNoChildren)                                           
user.dhill.Sent (\HasNoChildren)                                           
user.dhill.Sent Items (\HasNoChildren)                                     
user.dhill.Sent Messages (\HasNoChildren)                                  
user.dhill.Spam (\HasNoChildren)                                           
user.dhill.Trash (\HasNoChildren)                                          
user.dhill.Workopolis (\HasNoChildren)                                     


As the dhill user, it can't even see it's own mailbox and can't create it because it's already there:

localhost> cm user.dhill
createmailbox: Mailbox already exists

localhost> lm
Archive (\Subscribed \HasNoChildren)                                  
Archives (\Subscribed \HasNoChildren)                                 
DELETED.INBOX.Trash.INBOX^Deleted Messages.59529F68 (\HasNoChildren)  
DELETED.INBOX.Trash.asdf.59529F34 (\HasNoChildren)                    
DELETED.INBOX.Trash.dhill.59529F37 (\HasNoChildren)                   
Junk (\Subscribed \HasNoChildren)                                     
Notes (\Subscribed \HasNoChildren)                                    
Sent (\Subscribed \HasNoChildren)

Comment 17 Jason Tibbitts 2017-06-27 21:15:26 UTC
OK, that overwrote my previous comments.

At this point I have no idea what you have added or removed from the configuration you posted previously.  It seems obvious that you have added "unixhierarchysep: 0" but I'm not sure if you added "altnamespace: 0" as you would have done if following the upgrade instructions.  I'm also not sure if you removed 'dhill' from the admins line as I indicated previously that you absolutely had to do.  But if you didn't then that would explain why the hierarchy is so messed up.

Really, providing complete information in your dialog which you have already started on the cyrus mailing list far more likely to produce a favorable result.  I'm out of guesses to give you.

Comment 18 David Hill 2017-06-27 21:24:18 UTC
Argh... man you're a genius.   I don't know why cyrus-imapd doesn't like having my user listed as an admin but this did it and my user can now see all folders.  That's still weird as I was able to see "shared folders" with that user instead of "none" at all.   Not only it was a bad idea, but it broke the cyrus-imapd expected behavior of working properly even though my user is admin.    Thank you buddy.  I think we can close this BZ now.

Comment 19 Jason Tibbitts 2017-06-27 21:35:57 UTC
Admins get to see the full namespace.  Users see a completely different namespace.  Admins don't have an INBOX.  The documentation is extremely clear that a user who receives mail should never be added to the admins line.  I did point that out in comment 5, but I figured maybe you had only added it temporarily in an attempt to debug things.


Note You need to log in before you can comment on or make changes to this bug.