By "passed to ivy", obviously I mean something like:

ant -Dlocal.mode=true ...

Related to Bug 1005971

Here my thoughts about this topic.

1) Using pattern-based repositories (like in comment #0) works only in some cases. Fedora supports more advanced features like artifact aliases, installing artifact into subdirectories, installing JARs with different names than module name (artifactId). This means that A custom Ivy resolver, which would fully understand Fedora layout, needs to be implemented.

2) Ivy is a library that does artifact resolution.  It can be used as Ant tasks, but it can also be used by other build systems. Enabling it to support local artifact resolution would not only help packages built with Ant, but also other build systems using Ivy.

3) Ivy modules could be dynamically mapped to Maven modules: organization to groupId, module to artifactId and so on. Even module dependencies can be mapped. This means that adding full support for Ivy wouldn't require infrastructure changes.

4) The only software that understands current repository layout used by Fedora is XMvn, which depends on Maven. This means that any package wanting to do local artifact resolution would need to have Maven installed. This may be not a problem at all, but I would rather try to attempt some refactoring to cut unneeded dependencies.

Suggestion; consider a command line property alias in order to avoid any project ivy setting patches.

For example, something like...

-Divy.xmvn.alias="ibiblio:xmvn"

Again, this might still necessitate a complementary property which defines a different resolver pattern to be used by the alias. Then, again the Fedora layout is somewhat well-defined...perhaps a series of Fedora artifact patterns could be tried.

(In reply to Pete MacKinnon from comment #4)
> Again, this might still necessitate a complementary property which defines a
> different resolver pattern to be used by the alias. Then, again the Fedora
> layout is somewhat well-defined...perhaps a series of Fedora artifact
> patterns could be tried.

Fedora repository structure is well defined, but its complexity is beyond capabilities of Ivy pattern-based resolver.

For example, how would parrern-based resolver possibly be able to resolve com.google.collections:google-collections to /usr/share/java/guava.jar?

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Reassigning back to myself.

Ivy Resolver was implemented in XMvn 1.5.0-SNAPSHOT.  It will be eventually integrated with Ivy somehow, but you can already try it out -- there are pre-release XMvn RPMs available at Jenkins.  You can use this repo:

[xmvn-jenkins]
name=xmvn-jenkins
baseurl=http://jenkins.cloud.fedoraproject.org/job/xmvn/ws/RPM/latest

To use it you can add this to your ivysettings.xml:

  <typedef name="xmvn" classname="org.fedoraproject.maven.connector.ivy.IvyResolver"/>
  <resolvers>
    <xmvn name="XMvn"/>
  </resolvers>

For SBT you can add this to .sbt file instead:

  externalResolvers := Seq(new sbt.RawRepository(new org.fedoraproject.maven.connector.ivy.IvyResolver))

(To use the resolver you may need to add the following extra dependencies to classpath: aether/api guava ivy maven/maven-model plexus-classworlds plexus-containers/plexus-container-default plexus/utils xbean/xbean-reflect xmvn/xmvn-connector xmvn/xmvn-core)

Any feedback is very welcome, it may speed things up.

My proposal of integrating this with Ivy is creating a system property (lets call it "ivy.mode") which would control in which mode Ivy works.  There would be three modes of operation:

1. Upstream mode, which mimics upstream behavior.  This mode is used when ivy.mode property is not defined, but can also be explicitly activated by setting ivy.mode to "upstream".

2. Local mode.  Activated by defining ivy.mode property to "local".  First tries local cache and then local system repository.

3. Bootstrap mode.  Activated by defining ivy.mode property to "bootstrap".  First tries local cache, then system local repository and then remote repositories (shared and public).  This mode is mostly useful in bootstrapping new software for which not all dependencies are yet available in system repository.

Hi Mikolaj,

Tried this out with my hive build. Ivy makes it difficult to inject the new resolver into the classpath so I resorted to:

  <classpath file="/usr/share/xmvn/lib/xmvn-core.jar"/>
  <classpath file="/usr/share/xmvn/lib/xmvn-connector.jar"/>
  <classpath file="/usr/share/xmvn/lib/aether_aether-api.jar"/>
  <classpath file="/usr/share/java/plexus-containers/plexus-container-default.jar"/>
  <classpath file="/usr/share/java/plexus-classworlds.jar"/>
  <classpath file="/usr/share/java/plexus/utils.jar"/>
  <classpath file="/usr/share/xmvn/lib/guava.jar"/>
  <typedef name="xmvn" classname="org.fedoraproject.maven.connector.ivy.IvyResolver"/>
  <resolvers>
    <xmvn name="XMvn"/>
  </resolvers>

But got this error:

Caused by: java.lang.IllegalArgumentException: bad method found for xmvn on class org.apache.ivy.core.settings.IvySettings
        at org.apache.ivy.util.Configurator.startCreateChild(Configurator.java:535)
        at org.apache.ivy.core.settings.XmlSettingsParser.inConfiguratorStarted(XmlSettingsParser.java:579)
        at org.apache.ivy.core.settings.XmlSettingsParser.startElement(XmlSettingsParser.java:201)
<snip>

In general, what do you see as the correct way to load the xmvn resolver and its deps into the Ivy classloader? The ivy <classpath> element only accepts a single file or url (no wildcards afaict). I suppose I could locally explode the xmvn jars and deps and pack them into one jar as part of the hive build, but that seems heinous.

(In reply to Pete MacKinnon from comment #9)
> Caused by: java.lang.IllegalArgumentException: bad method found for xmvn on
> class org.apache.ivy.core.settings.IvySettings

I never saw this.  I wonder if it has anything to do with incpomplete classpath.

> 
> In general, what do you see as the correct way to load the xmvn resolver and
> its deps into the Ivy classloader? The ivy <classpath> element only accepts
> a single file or url (no wildcards afaict). I suppose I could locally
> explode the xmvn jars and deps and pack them into one jar as part of the
> hive build, but that seems heinous.

For ant, this should do it (persistently): echo "aether/api guava ivy maven/maven-model plexus-classworlds plexus-containers/plexus-container-default plexus/utils xbean/xbean-reflect xmvn/xmvn-connector xmvn/xmvn-core" >/etc/ant.d/xmvn

or alternatively (for one session, or without root access): export OPT_JAR_LIST="aether/api guava ivy maven/maven-model plexus-classworlds plexus-containers/plexus-container-default plexus/utils xbean/xbean-reflect xmvn/xmvn-connector xmvn/xmvn-core"

Mikolaj, looks good so far:

# rpm -q xmvn maven-local
xmvn-1.5.0-0.9.git5b8a5e3.noarch
maven-local-3.4.2-1.fc21.noarch

<snip>
[ivy:resolve] :: loading settings :: file = /builddir/hive-0.12.0/ivy/ivysettings.xml
[ivy:resolve] :: resolving dependencies :: org.apache.hive#hive-shims;0.12.0
[ivy:resolve]   confs: [hadoop0.23.shim]
[ivy:resolve]   found org.apache.hadoop#hadoop-common;SYSTEM in XMvn
[ivy:resolve]   [SYSTEM] org.apache.hadoop#hadoop-common;2.2.0 (forced)
[ivy:resolve]   found org.apache.zookeeper#zookeeper-test;SYSTEM in XMvn
[ivy:resolve]   [SYSTEM] org.apache.zookeeper#zookeeper-test;any (forced)
[ivy:resolve]   found org.jboss.netty#netty;SYSTEM in XMvn
[ivy:resolve]   [SYSTEM] org.jboss.netty#netty;any (forced)
[ivy:resolve]   found com.jcraft#jzlib;SYSTEM in XMvn
[ivy:resolve]   [SYSTEM] com.jcraft#jzlib;any (forced)
[ivy:resolve]   found junit#junit;SYSTEM in XMvn
[ivy:resolve]   [SYSTEM] junit#junit;any (forced)
...

Implemented upstream in javapackages-tools 3.5.0.  To use this feature you need to install "ivy-local" package.

There are 3 modes of behaviour, controlled by setting "ivy.mode" property.

1) local mode, activated with -Divy.mode=local
First try Ivy local repository, then system repository through XMvn.

2) bootstrap mode, activated with -Divy.mode=bootstrap
First try Ivy local repository, then system repository through XMvn, finally public remote repositories (download artifacts from the Internet).  This mode is useful when building packages for which not all dependencies have yet been packaged for Fedora.

3) upstream mode, activate by default (-Divy.mode=upstream works too)
Mimic upstream behaviour (local first, then remote).  Never tries to resolve from system repos.

There is no need for custom settings or touching classpath.  Everything should work out of the box after defining "ivy.mode".

To summarize:
  BuildRequires: ivy-local
  ant -Divy.mode=local ...

Fixed in javapackages-tools-3.5.0-1

I believe that this feature is included in javapackages-tools-3.5.0-1,
which is available in Fedora Rawhide, so I am closing this bug now.

The build which includes the feature can be found at Koji:
http://koji.fedoraproject.org/koji/buildinfo?buildID=491550