Bug 1094454 - [RFE] Memcached should be able to be used as the cache store for the broker and console
Summary: [RFE] Memcached should be able to be used as the cache store for the broker a...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 2.0.0
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: ---
Assignee: Timothy Williams
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-05-05 17:53 UTC by Timothy Williams
Modified: 2015-07-21 19:11 UTC (History)
8 users (show)

Fixed In Version: openshift-origin-broker-1.16.2.8-1, rubygem-openshift-origin-console-1.34.1.1-1,openshift-origin-console-1.16.3.6-1
Doc Type: Enhancement
Doc Text:
When attempting to use an alternative cache store for the broker or console, the Gemfile must be modified. The Gemfile is not marked as a configuration file in the respective RPMs, and any changes can be overwritten on updates. This enhancement enables additional gems to be loaded into the broker or console Rails environments by utilizing the ADDITIONAL_GEMS parameter in the /etc/openshift/broker.conf or /etc/openshift/console.conf file, respectively. The broker or console service must be restarted for the change to take effect. As a result, additional gems can be loaded into the broker or console environments without the need to modify Gemfiles and without losing this configuration change upon update or upgrade.
Clone Of:
Environment:
Last Closed: 2015-07-21 19:11:46 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 800093 0 None None None Never
Red Hat Product Errata RHBA-2015:1463 0 normal SHIPPED_LIVE Red Hat OpenShift Enterprise 2.2.6 bug fix and enhancement update 2015-07-21 23:11:33 UTC

Description Timothy Williams 2014-05-05 17:53:24 UTC
Description of problem:
The broker and console should be able to use memcached as the cache store. The default rails cache store creates a file for each unique request made to the application. Eventually, the rails cache store will use up all of the system's free inodes. Using memcached resolves this issue.

Currently, the configuration to use memcached requires changes to files that are not marked as configuration files in their respective rpm's. This means that they will be overwritten on a yum update.

See https://access.redhat.com/site/solutions/800093

Version-Release number of selected component (if applicable):
- 

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 2 Luke Meyer 2014-09-29 18:13:34 UTC
This ought to cause the broker to load the 'dalli' gem (if installed) without editing Gemfile.conf:

    # touch /etc/openshift/plugins.d/dalli.conf

Clearing or "cleaning" (remove expired) the cache periodically should also work around the issue.

Comment 3 Miciah Dashiel Butler Masters 2014-10-23 21:28:27 UTC
It seems that it is possible to use memcached with the broker as-is, following the instructions in the knowledge-base article and comments (i.e., modify CACHE_STORE in broker.conf and touch /etc/openshift/plugins.d/dalli.conf).  Is this solution for the broker side of the problem acceptable?

That leaves the management console.  We probably don't want to try to autoload required gems based on the CACHE_STORE setting because that would require parsing it and matching it with each rubygem we wanted to allow.  Instead, we could add something like an ADDITIONAL_RUBYGEMS setting to both console.conf and broker.conf and make Gemfile respect it.  Then enabling memcached would require just modifying the two settings (CACHE_STORE and ADDITIONAL_RUBYGEMS) in each of the two files (broker.conf and console.conf).  Would this be an acceptable solution?

Comment 4 Timothy Williams 2014-11-03 16:28:23 UTC
Miciah,

The issue with the broker setup is that it requires modification of the 'Gemfile'. When updates occur, the Gemfile is not marked as a configuration file and can be overwritten if any updates have been made. We either need to be able to do this without modifying the Gemfile.

Your solution to add an 'ADDITIONAL_RUBYGEMS' directive to broker.conf and console.conf appears to be spot on. This would solve the problem completely.

Comment 5 Timothy Williams 2014-11-03 19:52:34 UTC
Gave it a shot:

https://github.com/openshift/origin-server/pull/5920

Comment 6 openshift-github-bot 2015-05-15 22:57:44 UTC
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/c5bce4afae1aa461d805ba36336a2a54341d7e74
Specify additional gems to be loaded in the broker and console

Bug 1094454
Bugzilla link https://bugzilla.redhat.com/show_bug.cgi?id=1094454
Added ability to specify additional gems to be loaded in the broker and console Gemfiles.

Comment 9 Ma xiaoqiang 2015-05-19 07:26:42 UTC
Check on puddle [2.2.6/2015-05-18.1]

1. add 'nokogiri' package in broker.conf
ADDITIONAL_RUBYGEMS="nokogiri"

2. restart the broker
#yum install ruby193-rubygem-nokogiri   
#/etc/init.d/openshift-broker restart

3, Check the Gemfile.lock
vim /var/www/openshift/broker/Gemfile.lock
<--snip-->
nokogiri (1.5.11)

load nokogiri  without modifying the Gemfile


4. add 'nokogiri' package in console.conf
ADDITIONAL_RUBYGEMS="nokogiri"

5. restart the console
#/etc/init.d/openshift-console restart

6. Check the Gemfile.lock
# grep nokogiri /var/www/openshift/console/*

Not load the nokogiri package

Check the code on console
# vim /var/www/openshift/console/Gemfile
source 'https://rubygems.org'

# Fedora 19 splits psych out into its own gem.
if Gem::Specification.respond_to?(:find_all_by_name) and not Gem::Specification::find_all_by_name('psych').empty?
  gem 'psych'
end

gem 'openshift-origin-console', :require => 'console', :path => ENV['CONSOLE_PATH']

gem 'rake', '> 0.9.2'
gem 'pry', :require => 'pry' if ENV['PRY']
gem 'perftools.rb', :require => 'perftools' if ENV['PERFTOOLS']
gem 'syslog-logger', '~> 1.6.8', :require => false


group :assets do
  gem 'compass-rails', '~> 1.0.3'
  gem 'sass-rails',    '~> 3.2.5'
  gem 'coffee-rails',  '~> 3.2.2'
  gem 'jquery-rails',  '~> 3.1.0'
  gem 'uglifier',      '>= 1.2.6'
  gem 'therubyracer',  '>= 0.10'
  gem 'sass-twitter-bootstrap', '2.0.1'
  gem 'minitest',      '>= 3.5.0'
end


The codes are not merged into /var/www/openshift/console/Gemfile file

Comment 10 Timothy Williams 2015-05-19 14:10:43 UTC
Below are the steps to properly test this. I apologize, I should have included these before the bug was set ON_QA:

1) Add the new gem(s) to the ADDITIONAL_RUBYGEMS directive in the broker.conf

2) Ensure the gem(s) are available by installing them.:
     # oo-exec-ruby gem install <gem_name>

2a) Resolve any dependencies when installing the gem(s). For example, the 'nokogiri' gem will require you install at least the following packages with yum:
     ruby193-ruby-devel
     gcc-c++
     libxml2-devel
     zlib-devel
     patch

3) Re-bundle the Gemfile for the broker
     # cd /var/www/openshift/broker
     # oo-exec-ruby bundle install --local

4) Check the Gemfile.lock for the additionaly rubygem previously added to the broker.conf

5) Restart the broker service
    # service openshift-broker restart

The same above steps can be run for the console using the console.conf and running the bundle command in the '/var/www/openshift/console' directory.

Comment 11 Ma xiaoqiang 2015-05-20 01:33:25 UTC
Please check the PR #5920. The codes are not merged in /var/www/openshift/console/Gemfile file.

Comment 12 openshift-github-bot 2015-05-20 16:33:55 UTC
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/6e4af869c246c4692bda332292c42eb0fbae1056
Specify addition rubygems in the openshift-console, rathen than the admin-console

Bug 1094454
A change made for 1094454 (pr 5920) allowed additional rubygems for the openshift management console, rather than the openshift console.

Comment 13 Timothy Williams 2015-05-20 16:34:59 UTC
Sorry about that.

https://github.com/openshift/origin-server/pull/6149 should resolve this.

Comment 14 Timothy Williams 2015-05-20 21:06:20 UTC
https://github.com/openshift/origin-server/pull/6150 submitted to resolve possible encoding issues.

Comment 15 openshift-github-bot 2015-05-21 18:21:45 UTC
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/6becd5ae49f195d80bfd712b917ebe488613315b
Read configuration files from Gemfile as a binary

Bug 1094454
https://bugzilla.redhat.com/show_bug.cgi?id=1094454
External data must be read in as binary to avoid encoding conflicts in some environments where the encoding of the configuration files does not match the default encoding for ruby.

Comment 16 Ma xiaoqiang 2015-05-22 03:18:06 UTC
Check on puddle [2.2.6/2015-05-21.1]

1. Add ADDITIONAL_RUBYGEMS="nokogiri" in configuration file
#echo 'ADDITIONAL_RUBYGEMS="nokogiri"' >> /etc/openshift/broker.conf
#echo 'ADDITIONAL_RUBYGEMS="nokogiri"' >> /etc/openshift/console.conf

2. Install the package
# oo-exec-ruby gem install nokogiri

3. restart the service
# /etc/init.d/openshift-console restart   
# /etc/init.d/openshift-broker restart 

4. Check the Gemfile.lock
# grep -r nokogiri /var/www/openshift/*
/var/www/openshift/broker/Gemfile.lock:    nokogiri (1.6.6.2)
/var/www/openshift/broker/Gemfile.lock:  nokogiri
/var/www/openshift/console/Gemfile.lock:    nokogiri (1.6.6.2)
/var/www/openshift/console/Gemfile.lock:  nokogiri

The package is loaded.

Comment 18 errata-xmlrpc 2015-07-21 19:11:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1463.html


Note You need to log in before you can comment on or make changes to this bug.