Bug 773061

Summary: Disk Source Synchronization Results in FileNotFoundException
Product: [Other] RHQ Project Reporter: Stefan Negrea <snegrea>
Component: ContentAssignee: Stefan Negrea <snegrea>
Status: CLOSED CURRENTRELEASE QA Contact: Mike Foley <mfoley>
Severity: high Docs Contact:
Priority: medium    
Version: 4.3CC: hrupp
Target Milestone: ---   
Target Release: RHQ 4.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 789027 (view as bug list) Environment:
Last Closed: 2013-09-01 10:14:36 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 758753, 789027    

Description Stefan Negrea 2012-01-10 19:40:32 UTC
Description of problem:
A disk source sync operation will result in FileNotFoundException if the parent folder contains one or more subfolders. The sync operation will attempt to import the subfolder as it is. The operation will fail because the import process is designed to import only individual files.


How reproducible:
Everytime

Steps to Reproduce:
1. Create a disk repository. 
2. Have a subfolder in the folder with artifacts
3. Start a repository sync operation.
  
Actual results:
FileNotFoundException because the plugin will attempt to import the subfolder as a proper package. 

Expected results:
The sync process succeeds and imports actual files and does not attempt to import full folders.


Additional info:
Will need to find requirements with regards to importing subfolder content. Either the subfolders are ignored or subfolders (and their subfolders) are parsed for more packages to be imported.


Stack trace from the failure:

Tue Jan 10 13:21:11 CST 2012: Start synchronization of Repository [Disk Repo]
Tue Jan 10 13:21:11 CST 2012: Getting currently known list of content source packages...
Tue Jan 10 13:21:25 CST 2012: SYNCHRONIZATION ERROR - STACK TRACE FOLLOWS:
java.lang.reflect.UndeclaredThrowableException
	at $Proxy682.synchronizePackages(Unknown Source)
	at org.rhq.enterprise.server.plugin.pc.content.sync.PackageSourceSynchronizer.synchronizePackageMetadata(PackageSourceSynchronizer.java:120)
	at org.rhq.enterprise.server.plugin.pc.content.ContentProviderManager.synchronizeRepo(ContentProviderManager.java:313)
	at org.rhq.enterprise.server.content.RepoManagerBean.internalSynchronizeRepos(RepoManagerBean.java:1243)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:112)
	at org.jboss.ejb3.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:166)
	at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32)
	at sun.reflect.GeneratedMethodAccessor130.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.jboss.ejb3.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:118)
	at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:77)
	at sun.reflect.GeneratedMethodAccessor129.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.jboss.ejb3.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:118)
	at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156)
	at sun.reflect.GeneratedMethodAccessor128.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.jboss.ejb3.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:118)
	at org.jboss.ejb3.interceptor.EJB3InterceptorsInterceptor.invoke(EJB3InterceptorsInterceptor.java:63)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.ejb3.entity.TransactionScopedEntityManagerInterceptor.invoke(TransactionScopedEntityManagerInterceptor.java:54)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.ejb3.AllowedOperationsInterceptor.invoke(AllowedOperationsInterceptor.java:47)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.aspects.tx.TxPolicy.invokeInNoTx(TxPolicy.java:66)
	at org.jboss.aspects.tx.TxInterceptor$Supports.invoke(TxInterceptor.java:144)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.aspects.tx.TxPropagationInterceptor.invoke(TxPropagationInterceptor.java:95)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.ejb3.stateless.StatelessInstanceInterceptor.invoke(StatelessInstanceInterceptor.java:62)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.aspects.security.AuthenticationInterceptor.invoke(AuthenticationInterceptor.java:77)
	at org.jboss.ejb3.security.Ejb3AuthenticationInterceptor.invoke(Ejb3AuthenticationInterceptor.java:110)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.ejb3.ENCPropagationInterceptor.invoke(ENCPropagationInterceptor.java:46)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.ejb3.asynchronous.AsynchronousInterceptor.invoke(AsynchronousInterceptor.java:106)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.ejb3.stateless.StatelessContainer.localInvoke(StatelessContainer.java:240)
	at org.jboss.ejb3.stateless.StatelessContainer.localInvoke(StatelessContainer.java:210)
	at org.jboss.ejb3.stateless.StatelessLocalProxy.invoke(StatelessLocalProxy.java:84)
	at $Proxy322.internalSynchronizeRepos(Unknown Source)
	at org.rhq.enterprise.server.scheduler.jobs.RepoSyncJob.sync(RepoSyncJob.java:218)
	at org.rhq.enterprise.server.scheduler.jobs.RepoSyncJob.execute(RepoSyncJob.java:98)
	at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.rhq.enterprise.server.plugin.pc.content.ContentProviderManager$IsolatedInvocationHandler.invoke(ContentProviderManager.java:852)
	... 53 more
Caused by: org.rhq.enterprise.server.plugin.pc.content.SyncException: Error digesting file
	at org.rhq.enterprise.server.plugins.disk.DiskSource.createPackage(DiskSource.java:241)
	at org.rhq.enterprise.server.plugins.disk.DiskSource.syncPackages(DiskSource.java:204)
	at org.rhq.enterprise.server.plugins.disk.DiskSource.synchronizePackages(DiskSource.java:136)
	... 58 more
Caused by: java.io.FileNotFoundException: /home/snegrea/backup/war/Disk Repo/test (Is a directory)
	at java.io.FileInputStream.open(Native Method)
	at java.io.FileInputStream.<init>(FileInputStream.java:120)
	at org.rhq.core.util.MessageDigestGenerator.calcDigest(MessageDigestGenerator.java:255)
	at org.rhq.core.util.MessageDigestGenerator.calcDigestString(MessageDigestGenerator.java:291)
	at org.rhq.enterprise.server.plugins.disk.DiskSource.createPackage(DiskSource.java:239)
	... 60 more

Download mode of NEVER for source [Backup War], skipping package bits sync for repo [Disk Repo]
Tue Jan 10 13:21:25 CST 2012: Repository [Disk Repo] completed syncing with no errors.

Comment 1 Stefan Negrea 2012-01-10 22:38:15 UTC
Updated the code to ignore subfolders inside the repo folder. This will prevent the exception. Traversing the tree of subfolders was not an option because it would have allowed overlapping imports of packages with sub-repos. 


The set of requirements going forward:
- The plugin searches for a subfolder that matches the repo name in the subfolder tree of the disk source root folder. The search process stops once the first folder with the repo name is found.
- Once the repo folder is found, the plugin imports only packages in that folder.
- Any subfolders in the repo folder are ignored and not traversed.

Comment 2 Mike Foley 2012-01-23 20:35:59 UTC
Setup:
1) RHQ server started
2) Two sample war archives (eg. sample1.war, sample2.war)

Stimulate:
1) Create the following folder tree:
      /rhqsources    <- folder
      /rhqsources/testrepo    <- folder
      /rhqsources/testrepo/subfolder    <- folder
      /rhqsources/testrepo/sample1.war    <- file
      /rhqsources/testrepo/subfolder/sample2.war    <-file
2) Navigate to Administration -> Content -> Content Sources -> Create New -> Local Disk Storage 
3) Fill in the fields required with the following info:
    a) Name of the sources should be testrepo
    b) Root directory should be /rhqsources
    c) Package type: JBossAS5 - Web Application (WAR) - WAR File
    d) Resource type: JBossAS5 - Web Application (WAR)
4) Save
5) Synchronize the repository

Verification steps:
1) Verify that after Step 6 there are packages in the synchornized repository
    a) only sample1.war is present
    b) sample2.war was not discovered
    c) there are no synchronization errors
2)  Look in the database, rhq_package_version and check display version and  version fields. Display version should have the old version content,  version should have the following format [sha256=abcd1234]

Comment 3 Stefan Negrea 2012-02-09 16:09:25 UTC
Some negative test cases:

1) Step 1, create /rhqresources/testrepo but leave it empty. No packages should
be discovered for the repo. 
2) Step 1, create two subfolders underneath /rhqresources/testrepo. Add
subpackages in each subfolder. Only packages directly underneath testrepo
folder should be discovered.
3) Step 3, name the resource testrepo3. There should be no packages discovered
since testrepo3 is not in the subdirectory structure of rhqresources.

Comment 5 Heiko W. Rupp 2013-09-01 10:14:36 UTC
Bulk closing of items that are on_qa and in old RHQ releases, which are out for a long time and where the issue has not been re-opened since.