Bug 901210 (JBPAPP6-1657)
| Summary: | [GSS] (6.3.0) Cleanup deploy directories - AS7-6031 | |||
|---|---|---|---|---|
| Product: | [JBoss] JBoss Enterprise Application Platform 6 | Reporter: | Shaun Appleton <sappleto> | |
| Component: | Other | Assignee: | Default User <jbpapp-maint> | |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | ||
| Severity: | high | Docs Contact: | ||
| Priority: | high | |||
| Version: | 6.0.0 | CC: | bmaxwell, brian.stansberry, cdewolf, dereed, dimitris, jaugustin2006, jawilson, jlivings, jnevill, kkhan, msadhukh, myarboro, pjelinek, pjindal, pkremens, rdickens, sappleto, sgilda, smumford, soenke.juergensen, trogers, yann.albou, yann-email | |
| Target Milestone: | DR5 | Keywords: | Reopened | |
| Target Release: | EAP 6.3.0 | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| URL: | http://jira.jboss.org/jira/browse/JBPAPP6-1657 | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | Bug Fix | ||
| Doc Text: |
This release of JBoss EAP 6.3 includes a fix that ensures files and directories created in the *JBOSS_HOME/tmp* and *JBOSS_HOME/tmp/vfs* folders are removed before they can interfere with newly (re)started EAP instances.
In previous versions of JBoss EAP, older files may have been left behind after a server is shutdown unexpectedly (as JBoss EAP removes files in *JBOSS_HOME/tmp* and *JBOSS_HOME/tmp/vfs* as part of the shutdown process).
The fix in this release provides a failsafe to mitigate that scenario. If a JBoss EAP 6.3 server does not shut down gracefully the server will not have an opportunity to clean up these temporary files. Upon restart however, the server now queries the above locations and, if files from a previous instance are present, it initiates a rename/remove process that allows the creation of fresh files for the new instance (the old directories are renamed so as to avoid interfering with newly created files). These processes happen in parallel.
Upon restarting JBoss EAP (either gracefully or otherwise) old temporary files are now removed (either at shutdown or restart), to ensure they do not take up unnecessary disk space.
*NOTE* Users should avoid using the *-Xrs* JAVA_OPT as this limits signals processing and can result in the size of the tmp/vfs directories continuing to grow.
|
Story Points: | --- | |
| Clone Of: | ||||
| : | 1078665 (view as bug list) | Environment: | ||
| Last Closed: | 2014-06-28 15:40:18 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1078665 | |||
|
Description
Shaun Appleton
2012-11-26 15:36:20 UTC
AS-6031 should be AS7-6031 This was resolved in AS 7.2 so should be included in the 6.1 ER1 build. This is still not fixed in EAP 6.1.0 ER4. Can not find changes from AS7-6031 in EAP's sources. tmp/vfs is not cleaned up during boot. AS7-6031 is now https://issues.jboss.org/browse/WFLY-1547 and this shows this was not fixed in EAP 6.1.0.GA can this be addressed for EAP 6.1.1? This is still not fixed in EAP 6.1.1 ER1. Tmp directories are still not cleaned up by EAP. So EAP 6.1.1. will be released with this known issue and it will be in the release notes? (In reply to Shaun Appleton from comment #11) > So EAP 6.1.1. will be released with this known issue and it will be in the > release notes? I just looked at the release notes it points back this page. Genius! *** Bug 978062 has been marked as a duplicate of this bug. *** jaikiran pai <jpai> made a comment on jira WFLY-1547 Shaun, thanks for testing this. Dimitris Andreadis <dimitris> made a comment on jira WFLY-1547 BTW, the reaper thread how does it differentiate old tmp content that can be removed from new content that might be created during the startup. Is it checking the date? jaikiran pai <jpai> made a comment on jira WFLY-1547 >> BTW, the reaper thread how does it differentiate old tmp content that can be removed from new content that might be created during the startup. Is it checking the date? Each temp subdirectory within the tmp/vfs directory belongs to a specific "provider type". The "provider type" relies on a VFS API to create that temp subdirectory content. Now when the server is restarted, the "provider type" code (from within WildFly, for example) will ask the VFS API to create a temp dir for it. The VFS API then checks if there's already a subdirectory for that "provider type" (left behind possibly from a previous run) and if there's one, it renames it to something else and submits a background task to delete the contents of the renamed dir. It then go ahead and creates a new temp dir for this "provider type" which the "provider type" can use to add new content. That way, the deletion of older content and addition of new content can happen parallely without interfering with one another. This code has the details https://github.com/jbossas/jboss-vfs/blob/master/src/main/java/org/jboss/vfs/TempFileProvider.java#L86 $JBOSS_HOME/standalone/tmp/vfs/ $JBOSS_HOME/domain/servers/$SERVER_NAME/tmp/vfs/ folders are now correctly cleaned up. Additional info: User shouldn't use "-Xrs" opt as this limits OS signals processing -> size of tmp/vfs keeps growing. Verified on EAP 6.2.0.ER5. Added Doc Text and marked for inclusion in EAP 6.2 Release Notes. Testing using EAP6.2.beta1 show a circumstance where the folder can grow: 1) start server -> creates tmp folder and add content 2) kill server -> remaining content in tmp, is accepted as it is a kill 3) start server 4) shutdown server using CLI What is seen is that: - a shutdown brings back the tmp folder size to the size before the corresponding start, so it's not growing, but - the trash from a kill is never removed, so a crash or kill increases the folder size. The workaround would be to remove the left over content manually or via a script. This tidy-up should be included in the code too. I'm re-opening because this is NOT fixed as per the release notes -- after a kill the temp directories are not deleted after a restart. The fix was only partially implemented. A new method was added to TempFileProvider#create with an extra "cleanExisting" boolean. This means *all* callers to TempFileProvider#create have to be modified to add an extra ", true" on the end. But this was only done in 2 out of 5 calls in the main EAP source, so the other 3 uses still leak memory on a kill. In particular, server/src/main/java/org/jboss/as/server/deployment/DeploymentMountProvider.java is the main user of the VFS temp directory and does not have the fix. Verified on EAP 6.3.0.DR5 jboss-eap-6.2.0.GA: - start standalone - deploy some content - kill -9 <standalone_PID>, start standalone, jboss-cli.sh -c shutdown $ du -hs jboss-eap-6.2/standalone/tmp/ 6.5M jboss-eap-6.2/standalone/tmp/ - start standalone, kill -9 <standalone_PID>, start standalone, jboss-cli.sh -c shutdown $ du -hs jboss-eap-6.2/standalone/tmp/ 13M jboss-eap-6.2/standalone/tmp/ ... size of tmp folder keeps growing jboss-eap-6.3.0.DR5: - start standalone - deploy some content - kill -9 <standalone_PID>, start standalone, jboss-cli.sh -c shutdown $ du -hs jboss-eap-6.3/standalone/tmp/ 48K jboss-eap-6.3/standalone/tmp/ - start standalone, kill -9 <standalone_PID>, start standalone, jboss-cli.sh -c shutdown $ du -hs jboss-eap-6.3/standalone/tmp/ 48K jboss-eap-6.3/standalone/tmp/ ... tmp folder is properly cleaned up Same behaviour for domain mode. *** Bug 1043909 has been marked as a duplicate of this bug. *** |