RDO tickets are now tracked in Jira https://issues.redhat.com/projects/RDO/issues/
Bug 1487324 - Move mailing lists from @redhat.com to @lists.rdoproject.org
Summary: Move mailing lists from @redhat.com to @lists.rdoproject.org
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: RDO
Classification: Community
Component: Infrastructure
Version: unspecified
Hardware: Unspecified
OS: Unspecified
medium
unspecified
Target Milestone: ---
: trunk
Assignee: Alan Pevec
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-08-31 15:58 UTC by Rich Bowen
Modified: 2024-02-15 14:43 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-11-06 03:01:53 UTC
Embargoed:


Attachments (Terms of Use)

Description Rich Bowen 2017-08-31 15:58:25 UTC
We wish to move our mailing lists from @redhat.com to @lists.rdoproject.org

Note that this project is already in process, and I am creating this ticket purely to have a single place to track, and a place to point people when I get questions as to the status.

The original request looked like this:

We want to create three lists: dev.org,
users.org, and newsletter

The first two will be populated with the complete list of current
subscribers to the existing rdo-list. We will then
immediately announce the switch, and give instructions for unsubscribing
from whichever list(s) one is not interested in remaining on.

The newsletter@ list will start with the current list of the existing
rdo-newsletter list.

The existing lists will be closed. rdo-list will receive a final message
pointing people to the new list, in the event that someone stumbles
across the archive in the future. The description of the list will also
be updated accordingly. The archive will remain in place.

Comment 1 Rich Bowen 2017-08-31 15:59:51 UTC
At this time, MM3 has been brought up at lists.rdoproject.org, however that's not responding right now.

When it was last up, it looked like all of the mailing list archives have been imported.

Credentials for the current list setup have been provided to Duck, for the purpose of getting the subscriber list and so on.

Comment 2 Rich Bowen 2017-08-31 16:01:27 UTC
There's also a ServiceNow ticket at https://redhat.service-now.com/surl.do?n=INC0588026 which tracks some of the components that IT has to do.

Comment 3 Marc Dequènes (Duck) 2017-09-01 06:12:23 UTC
About the blockers we had for the migration, please read below.

So, INC0588026 has recently unblocked, and I was able to get the files and they look good. I cannot confirm the migration of these settings yet because of the problem with the UI (more info below).

INC0576330, about DNS propagation problem, already affected the website migration, is still stalled.

INC0581344, about DNS updates needed for this migration and future ones is still unassigned.

INC0588889 about accessing the ML mboxes is ongoing and there is a workaround, so not blocking anymore.

INC0587344 about SPF rejection exemption should not be a problem as this server is not supposed to handle @rdoproject.org aliases, only MLs. We would be able to use PostSRSd if this is needed in the future (on a separate mail server).

The list admin passwords to be able to fetch the mbox file were provided by Rich, and I was able to import the lists archives.

A way to import a list subscribers without the archives is not possible, as they are discovered during the import. I found meddling into the database was deemed to risky. Rich said having the same base archives for dev@ and users@ is acceptable so we'll go for it.

There were problems while experimenting archives import due to MM3 unable to clean the previous import completely and upstream gave information to work around the problem.

There was a problem with wrongly formated dates which were accepted by MM2 and now rejected by MM3. The mbox file is not fixed before being imported.

Bugs in the various Ansible rules were also fixed along the way.

Comment 4 Marc Dequènes (Duck) 2017-09-01 06:15:51 UTC
As for the problem with the web UI. There is no clear error, just a timeout and no crash. The external repositories dragged different version of some packages, including Python and Django, and it may have caused some problems. I'm still looking into it.

Comment 5 Marc Dequènes (Duck) 2017-09-01 06:17:46 UTC
We already setup external auth but I think I forgot to say Fedora auth is also possible; so if you're interested in activating this feature, just tell me, it's trivial to do so.

Comment 6 Marc Dequènes (Duck) 2017-09-05 18:55:51 UTC
I've got difficulties debugging the UI problem. It seems related to WSGI in some way as I am able to make it work with the debug webserver. I asked Misc to help me.

So what about the Fedora auth?

Comment 7 Marc Dequènes (Duck) 2017-09-06 10:45:04 UTC
Misc helped me debug and find the problem. The python2-numpy is used indirectly by Mailman 3 and was causing problem. This is due to the version difference with packages available in the openstack repository. Downgrading this package as well as its rdep python-networkx solved the problem. Nevertheless with each release of openstack we'll get new version and this will break things. I don't think this machine needs all the currently activated repositories and I there's no way to ensure the software will work properly. There are many other version differences and we may discover new bugs in the future. We then need to discuss this and adapt the Ansible rules accordingly. This is a BLOCKER for the migration.

INC0588026 is not blocking anymore. The data I got were fine. Although there was a few problems to solve to get it imported as MM2 is lax about the value while MM3 is not. I made a script to fix the settings and was able to import them and they show up in the interface.

I pingued all other IT tickets. As said in the former mail thread, if you have any contact to help push these tickets forward, please help.

Comment 8 Marc Dequènes (Duck) 2017-09-06 10:55:40 UTC
Also, even if we would probably have to reinstall the whole machine to start over cleanly after the debug and the repository problem, now that the web UI is working again, would you all please test it, familiarize with it, check the settings?

Comment 9 Marc Dequènes (Duck) 2017-09-08 05:02:10 UTC
Could you please give me news and update the ticket with your comments with you had a look at the new UI and the state of the list sync?

Could you reply to the question about Fedora auth too?

Also if you give me your handle I can add you to the system's super-admin so you have a deeper look at the settings.

Comment 10 Marc Dequènes (Duck) 2017-09-08 10:00:34 UTC
To get back on the extra repositories problem, we have another unexpected failure: cerbot, the tool for Let's Encrypt, is broken:
Traceback (most recent call last):
  File "/bin/certbot", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2927, in <module>
    @_call_aside
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2913, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2940, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 637, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 650, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 834, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (cryptography 1.3.1 (/usr/lib64/python2.7/site-packages), Requirement.parse('cryptography>=1.3.4'), set(['PyOpenSSL']))

This is due to the @centos-openstack-ocata repository bringing in a new version of pyOpenSSL. This library needs a newer python2-cryptography which is not provided by any repository.

Comment 11 Rich Bowen 2017-09-08 15:28:47 UTC
Re: Fedora auth, I'd say no, since we don't support Fedora at this time, so that's probably not useful.

Comment 12 Rich Bowen 2017-09-08 15:31:26 UTC
I'm unable to log into the UI at this time, but my login is my Github rbowen account.

Comment 13 Rich Bowen 2017-09-08 15:44:35 UTC
Other moderators on the dev and users lists should be the same as now: 

apevec
hbrock
hguemar
rbowen

Comment 14 Rich Bowen 2017-09-08 15:57:48 UTC
Sorry, that's my rbowen Goggle auth, not Github. However, I am able to manage the lists with my current auth.

Comment 15 Rich Bowen 2017-09-08 15:58:49 UTC
I have updated the list descriptions, and the moderator lists, on the dev and users lists.

Comment 16 Rich Bowen 2017-09-08 16:11:38 UTC
As far as I can tell, everything looks fine.

There's no obvious way to get a subscriber count in the UI to see if all the subscribers copied over. However a CSV export seems to have the correct number of lines.

It looks like the most recent messages are missing, but I suppose you already knew that.

Comment 17 Marc Dequènes (Duck) 2017-09-11 01:21:51 UTC
In the Ansible rules I'm giving admin access to a handle named "rbowen", but that was just a guess. I don't know which account will use the other ones. Also is does not always match the part before @redhat.com. Another things is, I cannot give access to accounts not created yet.

I cannot find a way to get the count except using the CSV export or looking into the database either.

As for the recent messages, they flow into the RH system, not this one, so I have to do manual operations to have them imported.

Comment 18 Marc Dequènes (Duck) 2017-09-13 03:12:40 UTC
While attempting to migrate the oVirt mailing-lists I found out a nasty bug and filed a BR: https://github.com/mailman/mailman/issues/152
As mails are lost because of it, this is a BLOCKER.

As for the problem with the repositories, this is complex, David is notified, and I filed a BR to look for a solution together: https://github.com/rdo-infra/rdo-infra-playbooks/issues/2

Comment 19 Marc Dequènes (Duck) 2017-09-19 15:59:06 UTC
As for INC0576330, the reply from IT is:
« The issue is we have a large legacy DNS implementation. It was done a LONG time ago and to be truthful, some of the methods we have used barely make sense to me now.
We will have to consider some large changes to resolve this particular problem. »
So we can play with the list of NS servers in the zone SOA but in the end it makes migrations painful, probably longer, because we have no way to control propagation properly. There is a discussion on the Community Cage about this issue and possibly creating a DNS network.

As for INC0581344, It works now. It seems someone already asked the same change in parallel, so I guess communication needs to be improved to avoid duplicate work (both for us ad IT).

Comment 20 Marc Dequènes (Duck) 2017-09-19 16:00:50 UTC
Remaining issues:
  - problem with the repositories (https://github.com/rdo-infra/rdo-infra-playbooks/issues/2)
  - bug loosing mails (https://github.com/mailman/mailman/issues/152)

Comment 21 Marc Dequènes (Duck) 2017-09-19 16:01:48 UTC
(In reply to Marc Dequènes (Duck) from comment #20)
> Remaining issues:
>   - problem with the repositories
> (https://github.com/rdo-infra/rdo-infra-playbooks/issues/2)
>   - bug loosing mails (https://github.com/mailman/mailman/issues/152)

I mean the blockers, not that the rest do not need some care.

Comment 22 Marc Dequènes (Duck) 2017-09-27 05:24:28 UTC
So I've having problems getting people to help me on the repositories problem. It seems with the CentOS 7.4 release the problem with Let's Encrypt is gone but still the activation of other repositories is a problem to the stability of Mailman. I suggested a course of action but I'd like involved people's agreement and help before going forward.

I did not have time to dig further on the Mailman bug.

Comment 23 Rich Bowen 2017-10-02 18:42:56 UTC
I'm unclear on your first statement. Who is it that you need help from? Is it people in the CentOS community, or IT, or somewhere else? Is this something I can help with?

Comment 24 Rich Bowen 2017-10-02 21:07:13 UTC
I'm also unclear what the problem with DNS is, after reading through stuff again.

Tell me if I'm understanding here:

As far as I can tell from where I'm sitting, https://lists.rdoproject.org/ is complete and functional, and has been since September 5th. We just need to

1) Point the lists.rdoproject.org MX record to the right place
2) Close new messages to the old lists
3) Re-run the import to catch up
4) Add an auto-response on the old list, telling people to move to the new list

Yes?

What am I missing?

Trying to follow https://github.com/rdo-infra/rdo-infra-playbooks/issues/2 it seems very quickly to not have anything to do with the mailing lists, so I must be missing something.

And the "losing mail" bug doesn't seem relevant, as all of the imported archives look complete. Right?

Comment 25 Marc Dequènes (Duck) 2017-10-03 03:04:32 UTC
You're reading far too fast.

The problems with the repositories is not directly related to Mailman indeed, but it made the web interface totally broken, the let's encrypt script was also broken at some point too, and maybe other things.

The loosing mail problem is absolutely unrelated to import, it is related to normal activity. When the mails flow into the system, some may be lost between Postfix and the Mailman routing daemon. It is not delayed or mildly annoying, it is silently and irremediably destroyed.

I am as frustrated as you are, be sure of it. I'm working on the first problem, but I currently do not have enough time to dig more into the second one.

As for help, I asked for it during the meetings, and I got some replies and a plan is slowly drafting for the first problem. I reported the second problem to Mailman upstream and I hoped such a nasty problem would draw their attention but I was wrong. If we could ping the Red Hat people working on it (like abompard) this would be nice. I'm just feeling reporting problems to Mailman is seen as a nuisance, and hardly get any reply unless I got Quaid to fire-up some mail for me, so I'm a it at a loss here.

Comment 26 Marc Dequènes (Duck) 2017-10-03 05:00:19 UTC
There's progress on the repository problem (see ticket). We would need to be sure our future installation do not have the problem. The web server was provisioned in the same period and is affected too for example. I made the server clean, upgraded to CentOS 7.4 btw, rebooted, and I'm checking things. At the moment it seems to be encouraging as Let's Encrypt and the Mailman UI seem to work well. I'm testing this more in depth and will check the Ansible rules diff too.

Comment 27 Rich Bowen 2017-10-03 15:15:20 UTC
Awesome. Thanks for the clarification. That's very helpful.

Comment 28 Karsten Wade 2017-10-06 04:22:26 UTC
Looking back through this bug and knowing I haven't been part of the entire discussion, I wonder if Rich can clarify some points about requirements.

The goal of using lists.rdoproject.org is the main effect requested, yes? I.e. the most important thing to get done?

What is the prioritization on using MM3 features? Could you wait to get MM3 features?

Thus, what if we migrated in two steps?

I.e.:

1. Move to the new domain as an MM2 server
2. Resolve the MM3 bugs
3. Migrate lists.rdoproject.org from MM2 to MM3 [1]

We are reaching out directly to the upstream maintainer outside of the bug report Duck filed.

While we cannot give an ETA on when that bug will be fixed, can we give an ETA on when we could move an MM2 setup to an MM2 instance at lists.rdoproject.org?

footnotes
=========

[1] For end-users, I can see a single migration might be more desirable. OTOH, if people want to touch their new account to adjust settings, the MM2 interface is what they used before, might be less jarring.

Comment 29 Rich Bowen 2017-10-09 13:05:26 UTC
So, to refocus: There are two goals to this entire project:

1) Split the mailing list into users@ and dev@ so that users feel more comfortable asking their beginner questions.

2) Provide a forum-like interface to the mailing list to lower the bar to entry for people that want to participate in the lists.

#1 can be provided by any mailing list server. 

#2 was the motivation for MM3, because Hyperkitty. Related, I'm not a fan of Hyperkitty, as I've mentioned once or twice. #2 could also be provided by an installation of Apache PonyMail running alongside MM2, if desired.

I understand the reluctance to stand up technology that we're unfamiliar with. Also, the ElasticSearch requirement may be a problem. I wouldn't know. But this is something we could do without changing to MM3, and continue to run the lists on supported IT infra, rather than running our own mailing list service.

Regardless, goal #1 is BY FAR the higher priority, and if we can get that more quickly by doing this in two steps, great, do it.

If this also gives more time to research PonyMail, then a second migration would not be necessary at all - we'd just stand up PonyMail beside it and get that new functionality "for free".

Please see http://rdo.fosslists.org/list.html?rdo-list@redhat.com:2015-12 for an example deployment of PonyMail, mirroring the *existing* RDO mailing list. The mirroring was switched off about a year ago, so it's not active. This was set up by Daniel Gruno (the original author of PonyMail) to show me what it would look like.

Comment 30 Marc Dequènes (Duck) 2017-10-10 16:56:37 UTC
Well, most communities are already happy with hyperkitty, a more usable interface, a single account per-user instead of per-list, and social auth, so we focused on what's supported upstream first. We're not reluctant to use Rainbow mail or others, we just did not have time. It was already quite time consuming to work on having a working MM3 with working migrations. We're almost there, so it's slightly frustrating for us too.

As for the requirements of Ponymail they are much larger, the ML VM was sized small, and we would need to put some more in it. Also on a maintenance and security point of view we need proper Ansible rules, packages… and AFAIK there's some gaps we would have to fill. If we install something I guess you expect us to maintain it over time. We cannot be compared to the original author for that. It's interesting to see the sync could be done independently though, so we can prepare it aside from the mail handling server.

So the 2/3 steps plan would be: move to a new ML server with MM2 and do the split. This would also allow to use the RDO domain. Then later move to MM3 when it's fixed, and then add Ponymail when it's ready.

If you prefer to stay on IT's system and move things later, please tell me. I understood the branding was also an important point but you did not say anything about it in your sumup.

So I'll have a look how much time is needed to get step 1 so you can have a better view.

Comment 31 Rich Bowen 2017-10-10 17:45:15 UTC
+1

Make it so.

Getting the list split into dev@ and users@ is by far the highest priority here.

Comment 32 Karsten Wade 2017-10-10 20:48:48 UTC
We discussed this through in our meeting today and I'm proposing the following plan:


1. Migration from listman.r.c. to lists.rdop.o as MM2 => MM2.
   - Accomplishes separation of devel & users list
   - Accomplishes project-specific domain
2. Work on the MM3 mail drop bug with upstream
   - Now working in proper bug tracker
3. Within 7 to 10 days assess if keep to MM3 path or switch paths
   - Research & plan Apache Pony Mail config
4. If no to MM3, decide if we should enact Pony Mail plan
   - Works on top of $whatever
   - Can be re-used for other projects without migrating from MM2

Goals:

* By 12 Oct (Thu) Duck returns with a plan and timing for staying on MM2
* Plan migration to new domain for week of 16 to 20 Oct
* By 20 Oct decide on MM3 v Pony Mail
* By 27 Oct provided Pony Mail plan

Dependencies:

* RH DNS going smoothly (solved)
* MM3 fix (unsolved)
* Pony Mail config (unsolved)
* Alert community about two-stage migration
  - Domain switch most disruptive
  - Underlying tooling less disruptive

Comment 33 Rich Bowen 2017-10-10 20:55:59 UTC
So ... this looks ok, although I'm pretty much out of the office from October 13th to November 20th. So, yes, definitely anxious to see things moving along, but it's critical that there be lots of communication, not just with me, but directly with rdo-list, during this time, so that everyone is informed of what to expect.

Thanks.

Comment 34 Karsten Wade 2017-10-11 00:07:46 UTC
(In reply to Rich Bowen from comment #33)
> So ... this looks ok, although I'm pretty much out of the office from
> October 13th to November 20th. So, yes, definitely anxious to see things
> moving along, but it's critical that there be lots of communication, not
> just with me, but directly with rdo-list, during this time, so that everyone
> is informed of what to expect.
> 
> Thanks.

How can we best engage?

When there is an infra change that affects people / is visible, do the RDO infra folks handle their own communication?

Do you have any communication guides we should reference? (I'm not sure Duck has communicated on infra changes or outages to rdo-list. We might want to all work on a message?)

Comment 35 Marc Dequènes (Duck) 2017-10-11 09:23:43 UTC
As for communication, I can only give you example of previously done migration announcements I made and let you judge if any better wrapping is needed:
https://www.redhat.com/archives/rdo-list/2017-July/msg00013.html
https://lists.ovirt.org/pipermail/users/2017-September/084023.html

Comment 36 Marc Dequènes (Duck) 2017-10-11 09:44:06 UTC
As for the technical work:
- I asked for RDO cloud resources (a new VM), no reply yet, will raise this during the meeting
- I can switch the DNS RRs when the new VM is provisioned
- contrary to what I thought we do not have a ready to go MM2 Ansible role, but I'm currently preparing a new one based on the job made by Misc for Gluster and MI
- as for Postfix and integration of MM2 we already have it done
- the Ansible playbook should not be long
- the rest is content migration and testing, which take a bit of time if we want to be sure everything is fine, it should be much easier than MM3 though

So the plan would be to have it ready at the beginning of next week or earlier if I can, have some non-technical person have a look, and either do a fix round trip or schedule the switch. I think warning people at least one week before is necessary. Migrating a Monday seems to be a nice option as people are enjoying the end of their we or sleeping and I'm already awoken in the future.

Rich are you on an event of on PTO? Meaning are you able to have a look, tell if you're happy with the result and give a go?

As for my dates, I'm on event+PTO from 31 to 8, so it would be better to not schedule it before 23th to be there if a fix is needed post-migration for a full week.

Comment 37 Rich Bowen 2017-10-11 11:26:20 UTC
It's all events - CentOS Dojo at CERN, Open Source Summit Prague, OpenStack Summit in Sydney, SC17 in Denver. So I'll be "at work" but those events - in particular the OpenStack one - tend to be 12+ hour work days on the floor and email gets ignored for a few days.

I'm out Friday Oct 13 this week, and then next week I'm out from Tuesday on, with intermittent email access.

As for infra outages, for something lower impact, like a server reboot or whatever, we usually give 24 hours notice when we can. For something as high impact as email, which has the potential to halt the project entirely, I think 72 hours is probably reasonable.

Comment 38 Marc Dequènes (Duck) 2017-10-11 16:28:06 UTC
OK.

So a quick status update (just pasting my notes):
        - [-] VM: extended quota requested, waiting for ready notification
        - [W] Ansible rules: done, being tested
        - [T] migration script
        - [W] backup: asked for info, backup on backup.rdoproject.org possible
        - [B] supervision: would have to wait for the opstools/roles problem to be solved
        - [W] doc update: https://github.com/redhat-openstack/website/pull/1088
        - [W] announcement message: being discussed

Comment 39 Marc Dequènes (Duck) 2017-10-12 08:34:13 UTC
So the Ansible rules are going well. I'm testing the installation (on a local VM as I still don't have the final one) and looking into the migration.

I hit an Ansible bug (https://github.com/ansible/ansible/issues/26294) but made a workaround.

Any comment about the announcement message style?

Comment 40 Marc Dequènes (Duck) 2017-10-12 17:45:47 UTC
I've finished the Ansibke rules and migrations scripts. Some parameters in the configuration needed to be adapted to the new environment and list names (MM3 takes care of it while importing but not MM2).

So I'm ready to deploy, test mail routing and ask a test user to give feedback once I have the ACK to create the VM.

Comment 41 Marc Dequènes (Duck) 2017-10-16 08:29:50 UTC
I got the GO for creating the new VM. It is deployed and I imported the lists.

Now I'm having difficulties testing the mail routing because of INC0576426 and pinged IT about it.

I've had access to the backup machine and I'm looking into the setup.

Comment 42 Marc Dequènes (Duck) 2017-10-16 08:42:30 UTC
Btw, secondary MX are necessary to unsure we don't loose mail when the list VM is under maintenance or is having difficulties. I did not want to ask IT anything more. OSCI is providing this service so I configured it on both side. I'm updating the pending doc change accordingly.

Comment 43 Marc Dequènes (Duck) 2017-10-16 18:07:09 UTC
The backup is setup. At the moment it is also affected by the DNS problem.

As for the DNS problem itself, ns3.redhat.com is still publishing the old data. IT replied and they are working on a fix.

Comment 44 Stormy 2017-10-16 21:33:39 UTC
I just want to make sure we still have a need to host these mailing lists in the community cage. We've been working on this for a long time so I want to make sure now that some things have changed, that we are still doing the right thing.

My understanding:
* These mailing lists are currently supported by RH IT.
* We want to move them to a new domain name, rdoproject.org
* We want to add a forum ui with a tool like Hyperkitty or Apache Ponymail.
* Originally the Community Cage team recommended that we use Hyperkitty and that required a move to MM3. For various reasons (bugs we can't resolve), we are no longer moving to MM3.
* We still want to move to the new domain and add a forum ui (Hyperkitty or Ponymail - we are open to informed recommendations!) 

Should we still be hosting in the Community Cage, is that a requirement for domain names or for the forum ui? Or can we continue to host with RH IT? I just want to make sure we don't make a choice (based on dependencies that might not exist any more) that is harder to maintain long term.

Thanks!

Comment 45 Karsten Wade 2017-10-16 22:22:51 UTC
(In reply to Stormy from comment #44)
> I just want to make sure we still have a need to host these mailing lists in
> the community cage. We've been working on this for a long time so I want to
> make sure now that some things have changed, that we are still doing the
> right thing.
> 
> My understanding:
> * These mailing lists are currently supported by RH IT.

Yes, and can only be hosted on the redhat.com domain.

Red Hat IT Mailman support is rock-solid, I've used them many times over the last few decades. But there are hard limits to what can and cannot be done there.

> * We want to move them to a new domain name, rdoproject.org

lists.rdoproject.org to be exact, which cannot easily be hosted by Red Hat IT. (I've not tried that in a long time, it's a dark path of unknown length.)

> * We want to add a forum ui with a tool like Hyperkitty or Apache Ponymail.
> * Originally the Community Cage team recommended that we use Hyperkitty and
> that required a move to MM3. For various reasons (bugs we can't resolve), we
> are no longer moving to MM3.

This is not the plan of record, see comment #32 above.

We are moving in stages that support the original goals. First we move the domain and split the mailing lists (latter being original motive AIUI) while staying on Mailman 2. Thus the most disruptive part for the community doesn't involve a tool change. At the same time we evaluate Apache Ponymail while we work with Mailman upstream now that we have an actual connection there. Adding the forum UI then becomes an added feature separated from the disruptive move.

I'm sorry we didn't pursue this path to start, it seemed straightforward to do move actions at the same time. In the future I would still plan the same approach, but provide an escape hatch early on to go with the backup plan.

> * We still want to move to the new domain and add a forum ui (Hyperkitty or
> Ponymail - we are open to informed recommendations!) 
> 
> Should we still be hosting in the Community Cage, is that a requirement for
> domain names or for the forum ui? 

Hosting outside of redhat.com is a requirement since they don't easily provide hosting for new community domains and specific software solutions. For the forum UI (Ponymail or Hyperkitty) to be on the same sub-domain as the mailing lists it must be self-hosted.

> Or can we continue to host with RH IT? I
> just want to make sure we don't make a choice (based on dependencies that
> might not exist any more) that is harder to maintain long term.

The hosting is going to be on a VM provided by RDO Cloud. This means the underlying virt host will have the support of the RDO Cloud NOC. (Using open source services provided by others as part of the community infra cage offering is SOP.)

Duck is an active member of the RDO Infra team and is working closely with that group to provide the virt hosting for all of this. The end result will be something that others in RDO infra can be empowered to help manage so the keys-and-configs to the mailing list server can be distributed across RDO infra and not be dependent on the OSAS ComInfra team. This arrangement is by design for ComInfra, i.e. it's how we intend things to work when embedded in an upstream community.

Comment 46 Marc Dequènes (Duck) 2017-10-17 04:14:14 UTC
So I confirm the ML is hosted in RDO Cloud. The only service hosted in the Cage is the shared mail servers (backup MX), used for OSCI, oVirt and other projects. As this service already exist, there is no extra maintenance needed by adding RDO.

The deployment rules are currently hosted in our repository because I did not have the access at first and now because of a bug (explained earlier in this ticket but rather technical). They are planned to move in their repository. This is an important step to ensure we work together on this subject _as a team_ (the "empower" part Quaid said).

There is nothing else Cage-related.

Comment 47 Marc Dequènes (Duck) 2017-10-17 09:08:18 UTC
IT fixed the DNS problem, so I was able to go on.

I had a few problem with the Postfix<->Mailman bridge and SELinux, so I had to change the method used to communicate. It works now. I asked Misc to have a look at the postfix Ansible role changes just to be sure.

You can have a look here:
  https://lists.rdoproject.org/pipermail/taiste/2017-October/thread.html

The other list were imported too, so if anyone can check if I forgot something, it would be nice. I adapted the configuration in the import script but I may have missed something.

After this feedback we can plan on a migration, next Tuesday or Wednesday mid-morning for example (in Japan, so most other people will be sleeping while I switch).

Comment 48 Stormy 2017-10-18 17:42:04 UTC
(In reply to Marc Dequènes (Duck) from comment #46)
> So I confirm the ML is hosted in RDO Cloud. The only service hosted in the
> Cage is the shared mail servers (backup MX), used for OSCI, oVirt and other
> projects. As this service already exist, there is no extra maintenance
> needed by adding RDO.
> 
> The deployment rules are currently hosted in our repository because I did
> not have the access at first and now because of a bug (explained earlier in
> this ticket but rather technical). They are planned to move in their
> repository. This is an important step to ensure we work together on this
> subject _as a team_ (the "empower" part Quaid said).
> 
> There is nothing else Cage-related.

So my questions was Cage/CommInfra/RDO vs RH IT. This solution is a lot more work for our team (CommInfra/RDO) so just making sure that it is required. It sounds like we did not ask RH IT if they could do this but we don't think they have ever done this.

Comment 49 Marc Dequènes (Duck) 2017-10-18 18:03:32 UTC
As for the first step, it's true IT can probably do it, not sure about doing it in a timely manner though.

As for providing Mailman 3 and PonyMail I'm pretty sure that would be impossible.

So that's some work yes, but once we move to Mailman 3 it would be better to avoid maintaining two solutions. We already provide several Mailman 3 instances so that would simplify things.

Comment 50 M. Scherer 2017-10-19 13:27:57 UTC
IT did for lists.projectatomic in the past. Based on my recollection of the process, it was a exception for which they weren't really staffed, and without a SME, so slightly painful on their side. It went ahead since it came from really above and with a really tight deadline. I doubt they are gonna do a 2nd time.

And yes, that would be purely mailman 2 for the foreseeable future.

Comment 51 Marc Dequènes (Duck) 2017-10-24 02:07:04 UTC
The old MLs are marked are obsolete with subscription and post moderation and a rejection message explaining the move.

The migration itself is done and all seem to work well. I posted a reply to the announcement. I'm watching the logs and I'm available via IRC and mail.

The documentation changes were merged.

Comment 52 Marc Dequènes (Duck) 2017-10-24 02:30:30 UTC
Btw, now that we manage the whole server, we have a site-wide password, additionally to the per-list ones. The password is encrypted with the Ansible rules using Vault, accessible when we move the work into the RDO repos; I can hand it over if needed in the meanwhile.

I would probably be better to change all passwords now. Also users@ and dev@ currently have the same because they originate from the same one and have the same base settings.

Comment 53 Rich Bowen 2017-10-25 08:33:06 UTC
Thanks. I'll coordinate with the other list owners on changing the password.

All seems well, and mail is flowing. I think we're all good. Thanks, all.

I presume we should create a new ticket in the new ticket tracker for the remainder of the project?

Comment 54 Marc Dequènes (Duck) 2017-10-25 08:48:57 UTC
Yes we can start afresh and reference this ticket.

It would be step 2, so migration to MM3.

And we can also track step 3 which is working on PonyMail on a third ticket.

Comment 55 Karsten Wade 2017-11-06 03:01:53 UTC
Closing this bug as the in-title task of migrating from @redhat.com to lists.rdoproject.org has been completed.

In this bug discussion it was decided to separate providing the Web forum UI/UX to a second phase. The second phase begins with providing two proof of concept tools (MM3 and Pony Mail), then conducting a TBD decision making process between the two. (The TBD decision making will be in an additional bug report.)

Follow-on bugs for the record.

MM3 proof of concept:

https://bugzilla.redhat.com/show_bug.cgi?id=1509773

Pony Mail POC:

https://bugzilla.redhat.com/show_bug.cgi?id=1509776


Note You need to log in before you can comment on or make changes to this bug.