Created attachment 555684 [details] Kickstarts filed before, after and post script example Description of problem: A recently updated spacewalk instance at version 1.2 was updated, step by step, to version 1.6 including each intermediate version along the way in order. No problems were experienced during update. We use incliuded "post" scripts in our kickstarts. When editing the contents of a post script, changes we make and save are not reflected in the generated kickstart. Even if we delete the post script no change occurs. We also use variables in our kickstarts. If we edit the values in the variables, the values of the variables DO change in the kickstart. These problems weren't present in our previous working 1.2 production environment prior to updating. Version-Release number of selected component (if applicable): 1.6 How reproducible: Every time Steps to Reproduce: 1. Open an kickstart with an existing "post" script in the "Scripts" tab 2. Edit the post script e.g. add "echo test" 2. view the kickstart file - the "echo test" command is not present 1. Open a kickstart profile with an existing post script 2) delete the post script 3) view the generated kickstart file, the command in the deleted post script are still present. Actual results: No change occurs in the kickstart file. Expected results: The kickstart file should change to reflect the changes made in the post script. E.g. if I add "echo test" to the "post" script the words 'echo test' should appear in the %post section of the kickstart. Similarly if I delete the post script, the %post section of the script should not contain the contents of the "post" script. Additional info: I have attached a tar ball with 3 files: ks1.txt is how the kickstart file does look, regardless of whether I delete teh post script or not. ks2.txt is how I think the kickstart file should look after I delete the "post" script. post.txt is the post script itself.
Nathan, I just did a change to a kickstart script and then deleted one of the kickstart scripts and the appropriate kickstart file was re-generated accordingly immediately after hitting the "Update Kickstart"/"Delete Kickstart Script". I tired it on both - Spacewalk 1.6 on postgresql and Spacewalk 1.7nightly on oracle. Are you sure your update went ok? Anything suspicious in the logs? What version of spacewalk-java do you use?
Well, the update went OK without producing any showstopper errors. FYI we are using an Oracle backend. Here's a list of java packages installed: tzdata-java-2011l-4.el5 java-1.6.0-openjdk-src-1.6.0.0-1.23.1.9.10.el5_7 java-1.6.0-openjdk-javadoc-1.6.0.0-1.23.1.9.10.el5_7 javasqlite-20090430-1.el5.1 spacewalk-java-config-1.6.101-1.el5 asm-javadoc-1.5.3-1jpp.ep1.1.el5.2 java-1.6.0-openjdk-devel-1.6.0.0-1.23.1.9.10.el5_7 spacewalk-java-oracle-1.6.101-1.el5 spacewalk-java-1.6.101-1.el5 java-1.4.2-gcj-compat-1.4.2.0-40jpp.115 spacewalk-java-lib-1.6.101-1.el5 java-1.6.0-openjdk-demo-1.6.0.0-1.23.1.9.10.el5_7 java-1.6.0-openjdk-1.6.0.0-1.23.1.9.10.el5_7 I've tried twice at updating spacewalk 1.2 to 1.6 and both produced problems. Trying to update a script produces a server error in one case. Profile->scripts... edit script, then update script... Server error Additionally updating kickstart snippets gives me an update "success" message, but these don't transfer over to the full kickstart. Would you like the schema update logs to look at? Where else may i find evidence of showstopper bugs during the update?
Here's a error readout from tomcat when updating the script fails:- 2012-02-20 15:12:52,731 [TP-Processor3] WARN org.apache.struts.action.RequestProcessor - Unhandled Exception thrown: class org.hibernate.PropertyValueException 2012-02-20 15:12:52,731 [TP-Processor3] ERROR com.redhat.rhn.frontend.servlets.SessionFilter - Error during transaction. Rolling back javax.servlet.ServletException: not-null property references a null or transient value: com.redhat.rhn.domain.kickstart.KickstartScript.scriptName at org.apache.struts.action.RequestProcessor.processException(RequestProcessor.java:535) at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:433) at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:237) at com.redhat.rhn.frontend.struts.RhnRequestProcessor.process(RhnRequestProcessor.java:99) at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196) at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432) at javax.servlet.http.HttpServlet.service(HttpServlet.java:710) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at com.redhat.rhn.frontend.servlets.AuthFilter.doFilter(AuthFilter.java:91) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:142) at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:58) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at com.redhat.rhn.frontend.servlets.LocalizedEnvironmentFilter.doFilter(LocalizedEnvironmentFilter.java:67) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at com.redhat.rhn.frontend.servlets.EnvironmentFilter.doFilter(EnvironmentFilter.java:108) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at com.redhat.rhn.frontend.servlets.SessionFilter.doFilter(SessionFilter.java:55) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at com.redhat.rhn.frontend.servlets.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:97) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151) at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200) at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291) at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:775) at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:704) at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:897) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:636) Caused by: org.hibernate.PropertyValueException: not-null property references a null or transient value: com.redhat.rhn.domain.kickstart.KickstartScript.scriptName at org.hibernate.engine.Nullability.checkNullability(Nullability.java:72) at org.hibernate.event.def.DefaultFlushEntityEventListener.scheduleUpdate(DefaultFlushEntityEventListener.java:270) at org.hibernate.event.def.DefaultFlushEntityEventListener.onFlushEntity(DefaultFlushEntityEventListener.java:128) at org.hibernate.event.def.AbstractFlushingEventListener.flushEntities(AbstractFlushingEventListener.java:196) at org.hibernate.event.def.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:76) at org.hibernate.event.def.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:35) at org.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:969) at org.hibernate.impl.SessionImpl.listCustomQuery(SessionImpl.java:1690) at org.hibernate.impl.AbstractSessionImpl.list(AbstractSessionImpl.java:142) at org.hibernate.impl.SQLQueryImpl.list(SQLQueryImpl.java:152) at com.redhat.rhn.domain.user.UserFactory.findRandomOrgAdmin(UserFactory.java:126) at com.redhat.rhn.manager.kickstart.KickstartFormatter.<init>(KickstartFormatter.java:186) at com.redhat.rhn.manager.kickstart.KickstartFormatter.<init>(KickstartFormatter.java:197) at com.redhat.rhn.domain.kickstart.KickstartFactory.saveKickstartData(KickstartFactory.java:370) at com.redhat.rhn.domain.kickstart.KickstartFactory.saveKickstartData(KickstartFactory.java:406) at com.redhat.rhn.manager.kickstart.BaseKickstartCommand.store(BaseKickstartCommand.java:92) at com.redhat.rhn.frontend.action.kickstart.BaseKickstartEditAction.execute(BaseKickstartEditAction.java:87) at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431) ... 40 more 2012-02-20 15:12:52,759 [TP-Processor3] ERROR org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/rhn].[action] - Servlet.service() for servlet action threw exception org.hibernate.PropertyValueException: not-null property references a null or transient value: com.redhat.rhn.domain.kickstart.KickstartScript.scriptName at org.hibernate.engine.Nullability.checkNullability(Nullability.java:72) at org.hibernate.event.def.DefaultFlushEntityEventListener.scheduleUpdate(DefaultFlushEntityEventListener.java:270) at org.hibernate.event.def.DefaultFlushEntityEventListener.onFlushEntity(DefaultFlushEntityEventListener.java:128) at org.hibernate.event.def.AbstractFlushingEventListener.flushEntities(AbstractFlushingEventListener.java:196) at org.hibernate.event.def.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:76) at org.hibernate.event.def.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:35) at org.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:969) at org.hibernate.impl.SessionImpl.listCustomQuery(SessionImpl.java:1690) at org.hibernate.impl.AbstractSessionImpl.list(AbstractSessionImpl.java:142) at org.hibernate.impl.SQLQueryImpl.list(SQLQueryImpl.java:152) at com.redhat.rhn.domain.user.UserFactory.findRandomOrgAdmin(UserFactory.java:126) at com.redhat.rhn.manager.kickstart.KickstartFormatter.<init>(KickstartFormatter.java:186) at com.redhat.rhn.manager.kickstart.KickstartFormatter.<init>(KickstartFormatter.java:197) at com.redhat.rhn.domain.kickstart.KickstartFactory.saveKickstartData(KickstartFactory.java:370) at com.redhat.rhn.domain.kickstart.KickstartFactory.saveKickstartData(KickstartFactory.java:406) at com.redhat.rhn.manager.kickstart.BaseKickstartCommand.store(BaseKickstartCommand.java:92) at com.redhat.rhn.frontend.action.kickstart.BaseKickstartEditAction.execute(BaseKickstartEditAction.java:87) at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431) at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:237) at com.redhat.rhn.frontend.struts.RhnRequestProcessor.process(RhnRequestProcessor.java:99) at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196) at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432) at javax.servlet.http.HttpServlet.service(HttpServlet.java:710) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at com.redhat.rhn.frontend.servlets.AuthFilter.doFilter(AuthFilter.java:91) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:142) at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:58) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at com.redhat.rhn.frontend.servlets.LocalizedEnvironmentFilter.doFilter(LocalizedEnvironmentFilter.java:67) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at com.redhat.rhn.frontend.servlets.EnvironmentFilter.doFilter(EnvironmentFilter.java:108) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at com.redhat.rhn.frontend.servlets.SessionFilter.doFilter(SessionFilter.java:55) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at com.redhat.rhn.frontend.servlets.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:97) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151) at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200) at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291) at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:775) at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:704) at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:897) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:636)
Why don't you update to spacewalk-java packages to version 1.6.102-1? That version definitely address the ISE you got and might address also the kickstart file not being re-generated. If you do not update everything, make sure you update at least to: spacewalk-java-1.6.102-1.el5 spacewalk-java-config-1.6.102-1.el5 spacewalk-java-lib-1.6.102-1.el5 spacewalk-java-oracle-1.6.102-1.el5 spacewalk-taskomatic-1.6.102-1.el5
Ok, this seems to work a lot better now. I ran a "yum upgrade" and a whole slew of new packages showed up including updates to the spacewalk 1.6 packages. This seems to have fixed the issue. Strange really, because I ran "yum upgrade" several times during the upgrade process to 1.6, and it's only with these new packages that it works.
Ok, now back to the original problem. In a kickstart profile "Scripts" tab, I can reference kickstart snippets with some code like: $SNIPPET('spacewalk/1/app-config) if the app-config kickstart snippet exists, then it gets expanded. If it doesn't, I see the $SNIPPET line unexpanded in the final kickstart. However, if I place something like echo "This is a test" in the script, then this doesn't come out in the final kickstart at all. Is this a bug or feature?
Some of the packages were pushed to spacewalk 1.6 repo post GA, addressing various issues. > However, if I place something like > echo "This is a test" > in the script, then this doesn't come out in the final kickstart at all. This would be a bug. However if the snippet stuff from ks scripts gets reflected into the ks file, simple echo command has to be reflected as well. When I try to reproduce it, it works as expected. I'd like to mention, that Bugzilla is a bug tracking tool, not a support interface.
(In reply to comment #6) > Ok, now back to the original problem. The original problem was that Changes made to a postscript are not being reflected in kickstart (copying from this bug's summary line). Is that fixed or not? > In a kickstart profile "Scripts" tab, I can reference kickstart snippets with > some code like: > > $SNIPPET('spacewalk/1/app-config) > > if the app-config kickstart snippet exists, then it gets expanded. If it > doesn't, I see the $SNIPPET line unexpanded in the final kickstart. I believe this is a cobbler, not Spacewalk issue. When you try to run # cobbler profile getks --name your-profile-name you should see the same behaviour.
The situation has improved somewhat after a full system upgrade of rpms, and most of the issues have disappeared (particularly the ISE).... however I am still seeing issues with snippets being referenced in kickstart scripts, and then not showing up in the final kickstart. So here's an update of what is now visibly causing problems: I have an issue where some snippets are being extrapolated into scripts, and other ones not being extrapolated and just sitting in the script as $SNIPPET('spacewalk/1/whatever'), even when the snippet exists. I do not know why some snippets appear and one in particular does not. Dropping simple commands like "echo test" in the script works fine now. I also notice that some commented out lines in the script do not appear in the final kickstart, but some do. Typically commented out line with one hash appear, but if the start of the line begins with more than one hash, it doesn't show up at all. cobbler produces the same results manually via the cli command.
In case you would write your own ks snippet files, check their permissions and selinux content, if they can be accessed by tomcat, if case they won't be expanded in the ks file. When using snippets in the ks scripts, make sure you check the "Template" checkbox, otherwise the snippets won't be expanded. Otherwise, please describe reproducer steps to get the wrong behavior you see, so we can restore and fix the issues.
Updated to Spacewalk 1.7 What I have noticed after testing (and this is repeatable), is that unescaped dollar symbols ($) prevent the kickstart snippet from appearing in the post script. Part of our script does the following:- DATE=`date +%Y%m%d-%H%M` cp -p /etc/passwd /etc/passwd-${DATE} if i fail to put a backslash before the $ sign on the second line, the entire snippet fails to appear in the kickstart. instead I see $SNIPPET('spacewalk/1/kssnippetname') in the completed kickstart Even with a backslash in front of the $, the command still doesn't work right. I end up with two files: - "passwd-" and "passwd-<day's date>" This becomes magnitutes worse should I try to add lines to /etc/shadow which include dollar symbols as part of the password.
As I already wrote in Comment#11, you have to have checked "Template" checkbox, when using snippets. Provided code appears in my ks file without any additional backslashes.
(In reply to comment #11) > > What I have noticed after testing (and this is repeatable), is that unescaped > dollar symbols ($) prevent the kickstart snippet from appearing in the post > script. This doller symbol -- is it in the post script, or in the snippet body? > Part of our script does the following:- When you say script -- do you mean post script? > DATE=`date +%Y%m%d-%H%M` > cp -p /etc/passwd /etc/passwd-${DATE} > > if i fail to put a backslash before the $ sign on the second line, the entire > snippet fails to appear in the kickstart. instead I see > $SNIPPET('spacewalk/1/kssnippetname') in the completed kickstart Do you say that if you have unescaped ${DATE} in your _post_ script, it prevents the $SNIPPET('spacewalk/1/kssnippetname') from being expended? Did you make sure you have the Template checked, as Tomas pointed out in comment 10 and comment 12? > Even with a backslash in front of the $, the command still doesn't work right. > I end up with two files: - "passwd-" and "passwd-<day's date>" This however shows that at least once the command worked right, doesn't it? > This becomes magnitutes worse should I try to add lines to /etc/shadow which > include dollar symbols as part of the password. Dollars are preserved just fine in our tests. This bugzilla has gone through too many different scenarios and issues and it's getting very close to CANTFIX.
I think this old spacewalk-list entry sums up my pain here. I found myself nodding while reading it. https://www.redhat.com/archives/spacewalk-list/2010-August/msg00033.html Additionally I don't see a "template checkbox" anywhere within Spacewalk 1.7, at least not on the Kickstart Snippet editor page, or in the text editor. I managed to "cheat" by escaping dollar symbols twice, but if raw/noraw works then I'll use that. I think i've been tripped up by this because I simply didn't know about the templating functions (would be nice if there was a button in the editor toolbar). I seem to believe that something changed with the way cobbler worked between 1.2 and 1.7 that's been causing us pain.
I found the template checkbox, and it is ticked. No problem there. Additionally I find that it is "#raw" and "#end raw" (with a space). The documentation is wrong. It is still confusing however. Consider the following:- #raw cat >> /etc/shadow << EOF oracle:$1$xp4C1B7u$jHHy/Tgr4PORXylKe8Hgw60:13839:0:99999:7::: #end raw Looks OK in the "Kickstart File' tab. but when the kickstart gets run on a new server, the password still gets mangled thanks to the dollar signs. I have to do the following in order to get it to render properly in the finished build:- #raw cat >> /etc/shadow << EOF oracle:\\$1\\$xp4C1B7u\\$jHHy/Tgr4PORXylKe8Hgw60:13839:0:99999:7::: #end raw because the #raw tag is completely ineffectual for what I want to do.
should be an EOF after those snippets, but even with an EOF I still can't get the code to produce correctly in the final build. So the problem is now essentially:- * looks like it's going to work from the profile's "Kickstart File" tab * Doesn't do what I thought it would do in final build
(In reply to comment #15) > I found the template checkbox, and it is ticked. No problem there. > > Additionally I find that it is "#raw" and "#end raw" (with a space). The > documentation is wrong. > > It is still confusing however. Consider the following:- > > #raw > cat >> /etc/shadow << EOF > oracle:$1$xp4C1B7u$jHHy/Tgr4PORXylKe8Hgw60:13839:0:99999:7::: > #end raw This is of course shell thing and has nothing to do with snippets. If you try cat >> /etc/shadow << EOF oracle:$1$xp4C1B7u$jHHy/Tgr4PORXylKe8Hgw60:13839:0:99999:7::: EOF in your bash it will be mangled as well. You need to use << 'EOF'.
I'm sorry but we cannot use bugzilla as a chat about all things Spacewalk -- please feel free to reach the community on the spacewalk-list for that. Closing this bugzilla as CANTFIX. If you feel there is a specific bug in the product, please file a new bugzilla -- one bugzilla per bug/problem and we will try to triage and address that issue.