Bug 789027 - Disk Source Synchronization Results in FileNotFoundException
Summary: Disk Source Synchronization Results in FileNotFoundException
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: RHQ Project
Classification: Other
Component: Content
Version: 4.3
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: ---
: JON 3.0.1
Assignee: Stefan Negrea
QA Contact: Mike Foley
URL:
Whiteboard:
Depends On: 773061
Blocks: 788629
TreeView+ depends on / blocked
 
Reported: 2012-02-09 16:10 UTC by Stefan Negrea
Modified: 2013-09-03 15:09 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of: 773061
Environment:
Last Closed: 2013-09-03 15:09:54 UTC
Embargoed:


Attachments (Terms of Use)

Description Stefan Negrea 2012-02-09 16:10:16 UTC
+++ This bug was initially created as a clone of Bug #773061 +++

Description of problem:
A disk source sync operation will result in FileNotFoundException if the parent folder contains one or more subfolders. The sync operation will attempt to import the subfolder as it is. The operation will fail because the import process is designed to import only individual files.


How reproducible:
Everytime

Steps to Reproduce:
1. Create a disk repository. 
2. Have a subfolder in the folder with artifacts
3. Start a repository sync operation.
  
Actual results:
FileNotFoundException because the plugin will attempt to import the subfolder as a proper package. 

Expected results:
The sync process succeeds and imports actual files and does not attempt to import full folders.


Additional info:
Will need to find requirements with regards to importing subfolder content. Either the subfolders are ignored or subfolders (and their subfolders) are parsed for more packages to be imported.


Stack trace from the failure:

Tue Jan 10 13:21:11 CST 2012: Start synchronization of Repository [Disk Repo]
Tue Jan 10 13:21:11 CST 2012: Getting currently known list of content source packages...
Tue Jan 10 13:21:25 CST 2012: SYNCHRONIZATION ERROR - STACK TRACE FOLLOWS:
java.lang.reflect.UndeclaredThrowableException
	at $Proxy682.synchronizePackages(Unknown Source)
	at org.rhq.enterprise.server.plugin.pc.content.sync.PackageSourceSynchronizer.synchronizePackageMetadata(PackageSourceSynchronizer.java:120)
	at org.rhq.enterprise.server.plugin.pc.content.ContentProviderManager.synchronizeRepo(ContentProviderManager.java:313)
	at org.rhq.enterprise.server.content.RepoManagerBean.internalSynchronizeRepos(RepoManagerBean.java:1243)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:112)
	at org.jboss.ejb3.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:166)
	at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32)
	at sun.reflect.GeneratedMethodAccessor130.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.jboss.ejb3.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:118)
	at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:77)
	at sun.reflect.GeneratedMethodAccessor129.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.jboss.ejb3.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:118)
	at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156)
	at sun.reflect.GeneratedMethodAccessor128.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.jboss.ejb3.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:118)
	at org.jboss.ejb3.interceptor.EJB3InterceptorsInterceptor.invoke(EJB3InterceptorsInterceptor.java:63)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.ejb3.entity.TransactionScopedEntityManagerInterceptor.invoke(TransactionScopedEntityManagerInterceptor.java:54)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.ejb3.AllowedOperationsInterceptor.invoke(AllowedOperationsInterceptor.java:47)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.aspects.tx.TxPolicy.invokeInNoTx(TxPolicy.java:66)
	at org.jboss.aspects.tx.TxInterceptor$Supports.invoke(TxInterceptor.java:144)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.aspects.tx.TxPropagationInterceptor.invoke(TxPropagationInterceptor.java:95)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.ejb3.stateless.StatelessInstanceInterceptor.invoke(StatelessInstanceInterceptor.java:62)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.aspects.security.AuthenticationInterceptor.invoke(AuthenticationInterceptor.java:77)
	at org.jboss.ejb3.security.Ejb3AuthenticationInterceptor.invoke(Ejb3AuthenticationInterceptor.java:110)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.ejb3.ENCPropagationInterceptor.invoke(ENCPropagationInterceptor.java:46)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.ejb3.asynchronous.AsynchronousInterceptor.invoke(AsynchronousInterceptor.java:106)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
	at org.jboss.ejb3.stateless.StatelessContainer.localInvoke(StatelessContainer.java:240)
	at org.jboss.ejb3.stateless.StatelessContainer.localInvoke(StatelessContainer.java:210)
	at org.jboss.ejb3.stateless.StatelessLocalProxy.invoke(StatelessLocalProxy.java:84)
	at $Proxy322.internalSynchronizeRepos(Unknown Source)
	at org.rhq.enterprise.server.scheduler.jobs.RepoSyncJob.sync(RepoSyncJob.java:218)
	at org.rhq.enterprise.server.scheduler.jobs.RepoSyncJob.execute(RepoSyncJob.java:98)
	at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.rhq.enterprise.server.plugin.pc.content.ContentProviderManager$IsolatedInvocationHandler.invoke(ContentProviderManager.java:852)
	... 53 more
Caused by: org.rhq.enterprise.server.plugin.pc.content.SyncException: Error digesting file
	at org.rhq.enterprise.server.plugins.disk.DiskSource.createPackage(DiskSource.java:241)
	at org.rhq.enterprise.server.plugins.disk.DiskSource.syncPackages(DiskSource.java:204)
	at org.rhq.enterprise.server.plugins.disk.DiskSource.synchronizePackages(DiskSource.java:136)
	... 58 more
Caused by: java.io.FileNotFoundException: /home/snegrea/backup/war/Disk Repo/test (Is a directory)
	at java.io.FileInputStream.open(Native Method)
	at java.io.FileInputStream.<init>(FileInputStream.java:120)
	at org.rhq.core.util.MessageDigestGenerator.calcDigest(MessageDigestGenerator.java:255)
	at org.rhq.core.util.MessageDigestGenerator.calcDigestString(MessageDigestGenerator.java:291)
	at org.rhq.enterprise.server.plugins.disk.DiskSource.createPackage(DiskSource.java:239)
	... 60 more

Download mode of NEVER for source [Backup War], skipping package bits sync for repo [Disk Repo]
Tue Jan 10 13:21:25 CST 2012: Repository [Disk Repo] completed syncing with no errors.

--- Additional comment from snegrea on 2012-01-10 17:38:15 EST ---

Updated the code to ignore subfolders inside the repo folder. This will prevent the exception. Traversing the tree of subfolders was not an option because it would have allowed overlapping imports of packages with sub-repos. 


The set of requirements going forward:
- The plugin searches for a subfolder that matches the repo name in the subfolder tree of the disk source root folder. The search process stops once the first folder with the repo name is found.
- Once the repo folder is found, the plugin imports only packages in that folder.
- Any subfolders in the repo folder are ignored and not traversed.

--- Additional comment from mfoley on 2012-01-23 15:35:59 EST ---


Setup:
1) RHQ server started
2) Two sample war archives (eg. sample1.war, sample2.war)

Stimulate:
1) Create the following folder tree:
      /rhqsources    <- folder
      /rhqsources/testrepo    <- folder
      /rhqsources/testrepo/subfolder    <- folder
      /rhqsources/testrepo/sample1.war    <- file
      /rhqsources/testrepo/subfolder/sample2.war    <-file
2) Navigate to Administration -> Content -> Content Sources -> Create New -> Local Disk Storage 
3) Fill in the fields required with the following info:
    a) Name of the sources should be testrepo
    b) Root directory should be /rhqsources
    c) Package type: JBossAS5 - Web Application (WAR) - WAR File
    d) Resource type: JBossAS5 - Web Application (WAR)
4) Save
5) Synchronize the repository

Verification steps:
1) Verify that after Step 6 there are packages in the synchornized repository
    a) only sample1.war is present
    b) sample2.war was not discovered
    c) there are no synchronization errors
2)  Look in the database, rhq_package_version and check display version and  version fields. Display version should have the old version content,  version should have the following format [sha256=abcd1234]

--- Additional comment from snegrea on 2012-02-09 11:09:25 EST ---

Some negative test cases:

1) Step 1, create /rhqresources/testrepo but leave it empty. No packages should
be discovered for the repo. 
2) Step 1, create two subfolders underneath /rhqresources/testrepo. Add
subpackages in each subfolder. Only packages directly underneath testrepo
folder should be discovered.
3) Step 3, name the resource testrepo3. There should be no packages discovered
since testrepo3 is not in the subdirectory structure of rhqresources.

Comment 2 Simeon Pinder 2012-02-10 22:44:52 UTC
Moving to ON_QA as new RC 4 available to test in:
https://brewweb.devel.redhat.com//buildinfo?buildID=198384

Comment 3 Sunil Kondkar 2012-02-14 10:37:37 UTC
Verified on Version: 3.0.1.GA RC4 (Build Number: dd8a001:c5270fb)

Performed verification steps and verified that only packages in the repo folder are imported. There is no exception after sync process.

Tue Feb 14 15:00:03 IST 2012: Start synchronization of Repository [testrepo]
Tue Feb 14 15:00:03 IST 2012: Getting currently known list of content source packages...
Tue Feb 14 15:00:03 IST 2012: Removing...0
Tue Feb 14 15:00:03 IST 2012: Adding...1
Tue Feb 14 15:00:03 IST 2012: Updating...0
Tue Feb 14 15:00:03 IST 2012: MERGE COMPLETE.

Download mode of NEVER for source [testrepo], skipping package bits sync for repo [testrepo]
Tue Feb 14 15:00:03 IST 2012: Repository [testrepo] completed syncing with no errors.

Verified the Display_version and version fields content in the rhq_package_version table. Also performed the negative test cases.

Comment 4 Heiko W. Rupp 2013-09-03 15:09:54 UTC
Bulk closing of old issues in VERIFIED state.


Note You need to log in before you can comment on or make changes to this bug.