Generate p2 composite repository metadata from Jenkins (Hudson) jobs on the fly

Serving p2 build artifacts from Jenkins is complicated if builds from multiple Jenkins jobs should be aggregated into a master p2 composite repository. A rather simple approach to this problem is listed below, that uses Apache to generate compositeArtifacts.xml and compositeContent.xml on the fly. We (vogella) uses this to generate repos for our various Jenkins jobs (e.g the content of compositeArtifacts.xml is generated).

• Install Apache and point DocumentRoot to Jenkin’s jobs/ directory
  • Enable cgi scripts in Apache
    • On Debian or Ubuntu this is `a2enmod cgid`
  • AllowOverride all for the jobs/ directory (needed in order for .htaccess to work)
• Clone https://github.com/lemmy/p2CompositeRepoGeneratorinto jobs/ and fix file ownership/permissions
  • Cloning into a non-empty directory isn’t allowed, thus move the content of p2CompositeRepoGenerator/ into jobs/ (including .git)
  • `chmod +x composite*.xml`
  • `chown jenkins:nogroup composite*.xml .htaccess p2.index`
• Symlinking composite*.xml from individual jobs generates a p2 composite repository for all builds of that Jenkins jobs (contrary to all builds of all jobs)
• Open a browser on the corresponding DocumentRoot url and check the content of compositeArtifacts.xml and compositeContent.xml
  • If no repositories show up, check that Jenkins even archives non-zipped p2 repository artifacts during the post build action
  • An internal server errors can best be diagnosed by opening the Apache error log
• The final directory structure should look similar to (the second part lists the structure for our I-build of the eclipse.platform.releng.aggregator) :

jenkins@build:~$ ls -lA jobs/ total 72 drwxr-xr-x 8 jenkins nogroup 4096 Mar 27 10:18 .git -rw-r–r– 1 jenkins nogroup 28 Mar 27 09:06 .gitignore -rw-r–r– 1 jenkins nogroup 402 Mar 27 09:02 .htaccess drwxr-xr-x 5 jenkins nogroup 4096 Mar 27 10:42 C-MASTER-e4tools-0.14 drwxr-xr-x 5 jenkins nogroup 4096 Mar 27 10:42 I-MASTER-eclipse.platform.releng.aggregator -rwxr-xr-x 1 jenkins nogroup 3192 Mar 27 10:18 compositeArtifacts.xml lrwxrwxrwx 1 jenkins nogroup 22 Mar 27 09:02 compositeContent.xml -> compositeArtifacts.xml -rw-r–r– 1 jenkins nogroup 129 Mar 27 09:02 p2.index

jenkins@build:~$ ls -lA jobs/I-MASTER-eclipse.platform.releng.aggregator/ total 68 drwxr-xr-x 5 jenkins nogroup 4096 Mar 27 10:32 builds lrwxrwxrwx 1 jenkins nogroup 44 Mar 26 13:14 compositeArtifacts.xml -> /var/lib/jenkins/jobs/compositeArtifacts.xml lrwxrwxrwx 1 jenkins nogroup 42 Mar 26 13:16 compositeContent.xml -> /var/lib/jenkins/jobs/compositeContent.xml -rw-r–r– 1 jenkins nogroup 6458 Mar 26 11:57 config.xml -rw-r–r– 1 jenkins nogroup 3 Mar 26 11:57 nextBuildNumber lrwxrwxrwx 1 jenkins nogroup 11 Mar 27 10:42 p2.index -> ../p2.index

---

Disclaimer: This is probably not the best idea for busy p2 repositories. The script a) scans the directory tree for every invocation and b) sets the timestamp to $now (messing up caches). b3aggr is a much better tool for frequently used repositories.