Building a headless RCP application with Tycho

Recently I got the request to create a “headless RCP” application from an existing Eclipse project. I was reading several posts on that and saw that a lot of people using the term “headless RCP”. First of all I have to say that “headless RCP” is a contradiction in itself. RCP means Rich Client Platform. And a rich client is typically characterized by having a graphical user interface. A headless application means to have an application with a command line interface. So the characteristic here is to have no graphical user interface. When people are talking about a “headless RCP” application, they mean to create a command line application based on code that is created for a RCP application, but without the GUI. And that actually means they want to create an OSGi application based on Equinox.

For such a scenario I typically would recommend to use bndtools or at least plain Java with the bnd Maven plugins. But there are scenarios where this is not possible, e.g. if your whole project is an Eclipse RCP project which currently forces you to use PDE tooling, and you only want to extract some parts/services to a command line tool. Well, one could also suggest to separate those parts to a separate workspace where bndtools is used and consume those parts in the RCP workspace. But that increases the complexity in the development environment, as you need to deal with two different toolings for one project.

In this blog post I will explain how to create a headless product out of an Eclipse RCP project (PDE based) and how to build it automatically with Tycho. And I will also show a nice benefit provided by the bnd Maven plugins on top of it.

Let’s start with the basics. A headless application provides functionality via command line. In an OSGi application that means to have some services that can be triggered on the command line. If your functionality is based on Eclipse Extension Points, I suggest to convert them to OSGi Declarative Services. This has several benefits, one of them is that the creation of a headless application is much easier. That said this tutorial is based on using OSGi Declarative Services. If you are not yet familiar with that, give my Getting Started with OSGi Declarative Services a try. I will use the basic bundles from the PDE variant for the headless product here.

Product Definition

For the automated product build with Tycho we need a product definition. Of course with some special configuration parameters as we actually do not have a product in Eclipse RCP terms.

• Create the product project
  • Main Menu -> File -> New -> Project -> General -> Project
  • Set name to org.fipro.headless.product
  • Ensure that the project is created in the same location as the other projects.
  • Click Finish
• Create a new product configuration
  • Right click on project -> New -> Product Configuration
  • Set the filename to org.fipro.headless.product
  • Select Create configuration file with basic settings
  • Click Finish
• Configure the product
  • Overview tab
    • ID = org.fipro.headless
    • Version = 1.0.0.qualifier
    • Uncheck The product includes native launcher artifacts
    • Leave Product and Application empty
      Product and Application are used in RCP products, and therefore not needed for a headless OSGi command line application.
  • This product configuration is based on: plug-ins
    Note:
    You can also create a product configuration that is based on features. For simplicity we use the simple plug-ins variant.
  • Contents tab
    • Add the following bundles/plug-ins:
    • Custom functionality
      • org.fipro.inverter.api
      • org.fipro.inverter.command
      • org.fipro.inverter.provider
    • OSGi console
      • org.apache.felix.gogo.command
      • org.apache.felix.gogo.runtime
      • org.apache.felix.gogo.shell
      • org.eclipse.equinox.console
    • Equinox OSGi Framework with Felix SCR for Declarative Services support
      • org.eclipse.osgi
      • org.eclipse.osgi.services
      • org.eclipse.osgi.util
      • org.apache.felix.scr
  • Configuration tab
    • Start Levels
      • org.apache.felix.scr, StartLevel = 0, Auto-Start = true
        This is necessary because Equinox has the policy to not automatically activate any bundle. Bundles are only activated if a class is directly requested from it. But the Service Component Runtime is never required directly, so without that setting, org.apache.felix.scr will never get activated.
    • Properties
      • eclipse.ignoreApp = true
        Tells Equinox to to skip trying to start an Eclipse application.
      • osgi.noShutdown = true
        The OSGi framework will not be shut down after the Eclipse application has ended. You can find further information about these properties in the Equinox Framework QuickStart Guide and the Eclipse Platform Help.

Note:
If you want to launch the application from within the IDE via the Overview tab -> Launch an Eclipse application, you need to provide the parameters as launching arguments instead of configuration properties. But running a command line application from within the IDE doesn’t make much sense. Either you need to pass the same command line parameter to process, or activate the OSGi console to be able to interact with the application. This should not be part of the final build result. But to verify the setup in advance you can add the following to the Launching tab:

• Program Arguments
  • -console
• VM Arguments
  • -Declipse.ignoreApp=true -Dosgi.noShutdown=true
    When adding the parameters in the Launching tab instead of the Configuration tab, the configurations are added to the eclipse.ini in the root folder, not to the config.ini in the configuration folder. When starting the application via jar, the eclipse.ini in the root folder is not inspected.

Tycho build

To build the product with Tycho, you don’t need any specific configuration. You simply build it by using the tycho-p2-repository-plugin and the tycho-p2-director-plugin, like you do with an Eclipse product. This is for example explained here.

Create a pom.xml file in org.fipro.headless.app:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>org.fipro</groupId>
    <artifactId>org.fipro.parent</artifactId>
    <version>1.0.0-SNAPSHOT</version>
  </parent>

  <groupId>org.fipro</groupId>
  <artifactId>org.fipro.headless</artifactId>
  <packaging>eclipse-repository</packaging>
  <version>1.0.0-SNAPSHOT</version>

  <build>
    <plugins>
      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>tycho-p2-repository-plugin</artifactId>
        <version>${tycho.version}</version>
        <configuration>
          <includeAllDependencies>true</includeAllDependencies>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>tycho-p2-director-plugin</artifactId>
        <version>${tycho.version}</version>
        <executions>
          <execution>
            <id>materialize-products</id>
            <goals>
              <goal>materialize-products</goal>
            </goals>
          </execution>
          <execution>
            <id>archive-products</id>
            <goals>
              <goal>archive-products</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

For more information about building with Tycho, have a look at the vogella Tycho tutorial.

Running the build via mvn clean verify should create the resulting product in the folder org.fipro.headless/target/products. The archive file org.fipro.headless-1.0.0-SNAPSHOT.zip contains the product artifacts and the p2 related artifacts created by the build process. For the headless application only the folders configuration and plugins are relevant, where configuration contains the config.ini with the necessary configuration attributes, and in the plugins folder you find all bundles that are part of the product.

Since we did not add a native launcher, the application can be started with the java command. Additionally we need to open the OSGi console, as we have no starter yet. From the parent folder above configuration and plugins execute the following command to start the application with a console (update the filename of org.eclipse.osgi bundle as this changes between Eclipse versions):

java -jar plugins/org.eclipse.osgi_3.15.100.v20191114-1701.jar -configuration ./configuration -console

The -configuration parameter tells the framework where it should look for the config.ini, the -console parameter opens the OSGi console.

You can now interact with the OSGi console and even start the “invert” command implemented in the Getting Started tutorial.

Native launcher

While the variant without a native launcher is better exchangeable between operating systems, it is not very comfortable to start from a users perspective. Of course you can also add a batch file for simplification, but Equinox also provides native launchers. So we will add native launchers to our product. This is fairly easy because you only need to check The product includes native launcher artifacts on the Overview tab of the product file and execute the build again.

The resulting product now also contains the following files:

• eclipse.exe
  Eclipse executable.
• eclipse.ini
  Configuration pointing to the launcher artifacts.
• eclipsec.exe
  Console optimized executable.
• org.eclipse.equinox.launcher artifacts in the plugins directory
  Native launcher artifacts.

You can find some more information on those files in the FAQ.

To start the application you can use the added executables:

eclipse.exe -console

or

eclipsec.exe -console

The main difference in first place is that eclipse.exe operates in a new shell, while eclipsec.exe stays in the same shell when opening the OSGi console. The FAQ says “On Windows, the eclipsec.exe console executable can be used for improved command line behavior.”.

Note:
You can change the name of the eclipse.exe file in the product configuration on the Launching tab by setting a Launcher Name. But this will not affect the eclipsec.exe.

Command line parameter

Starting a command line tool with an interactive OSGi console is typically not what people want. This is nice for debugging purposes, but not for productive use. In productive use you usually want to use some parameters on the command line and then process the inputs. In plain Java you take the arguments from the main() method and process them. But in an OSGi application you do not write a main() method. The framework launcher has the main() method. To start your application directly you therefore need to create some kind of starter that can inspect the launch arguments.

With OSGi Declarative Services the starter is an immediate component. That is a component that gets activated directly once all references are satisfied. To be able to inspect the command line parameters in an OSGi application, you need to know how the launcher that started it provides this information. The Equinox launcher for example provides this information via org.eclipse.osgi.service.environment.EnvironmentInfo which is provided as a service. That means you can add a @Reference for EnvironmentInfo in your declarative service, and once it is available the immediate component gets activated and the application starts.

Create new project org.fipro.headless.app

• Create the app project
  • Main Menu -> File -> New -> Plug-in Project
  • Set name to org.fipro.headless.app
• Create a package via right-click on src
  • Set name to org.fipro.headless.app
• Open the MANIFEST.MF file
  • Add the following to Imported Packages
    • org.osgi.service.component.annotations
      Remember to mark it as optional to avoid runtime dependencies to the annotations.
    • org.eclipse.osgi.service.environment
      To be able to consume the Equinox EnvironmentInfo.
    • org.fipro.inverter
      To be able to consume the functional services.
• Add org.fipro.headless.app to the Contents of the product definition.
• Add org.fipro.headless.app to the modules section of the pom.xml.

Create an immediate component with the name EquinoxStarter.

@Component(immediate = true)
public class EquinoxStarter {

    @Reference
    EnvironmentInfo environmentInfo;

    @Reference
    StringInverter inverter;

    @Activate
    void activate() {
        for (String arg : this.environmentInfo.getNonFrameworkArgs()) {
            System.out.println(inverter.invert(arg));
        }
    }
}

With the simple version above you will notice some issues if you are not specifying the -console parameter:

1. If you start the application via eclipse.exe with an additional parameter, the code will be executed, but you will not see any output.
2. If you start the application via eclipsec.exe with an additional parameter, you will see an output but the application will not finish.

If you pass the -console parameter, the output will be seen in both cases and the OSGi console opens immediately afterwards.

First let’s have a look why the application seem to hang when started via eclipsec.exe. The reason is simply that we configured osgi.noShutdown=true, which means the OSGi framework will not be shut down after the Eclipse application has ended. So the simple solution would be to specify osgi.noShutdown=false. The downside is that now using the -console parameter will not keep the OSGi console open, but close the application immediately. Also using eclipse.exe with the -console parameter will not keep the OSGi console open. So the configuration parameter osgi.noShutdown should be set dependent on whether an interactive mode via OSGi console should be supported or not.

If both variants should be supported osgi.noShutdown should be set to true and a check for the -console parameter in code needs to be added. If that parameter is not set, close the application via System.exit(0);.

-console is an Equinox framework parameter, so the check and the handling looks like this:

boolean isInteractive = Arrays
    .stream(environmentInfo.getFrameworkArgs())
    .anyMatch(arg -> "-console".equals(arg));

if (!isInteractive) {
    System.exit(0);
}

With the additional handling above, the application will stay open with an active OSGi console if -console is set, and it will close immediately if -console is not set.

The other issue we faced was that we did not see any output when using eclipse.exe. The reason is that the outputs are not sent to the executing command shell. And without specifying an additional parameter, the used command shell is not even opened. One option to handle this is to open the command shell and keep it open as long as a user input closes it again. The framework parameter is -consoleLog. And the check could be as simple as the following for example:

boolean showConsoleLog = Arrays
    .stream(environmentInfo.getFrameworkArgs())
    .anyMatch(arg -> "-consoleLog".equals(arg));

if (showConsoleLog) {
    System.out.println();
    System.out.println("***** Press Enter to exit *****");
    // just wait for a Enter
    try (BufferedReader reader = new BufferedReader(new InputStreamReader(System.in))) {
        reader.readLine();
    } catch (IOException e) {
        e.printStackTrace();
    }
}

With the -consoleLog handling, the following call will open a new shell that shows the result and waits for the user to press ENTER to close the shell and finish the application.

eclipse.exe test -consoleLog

bnd export

Although these results are already pretty nice, it can be even better. With bnd you are able to create a single executable jar that starts the OSGi application. This makes it easier to distribute the command line application. And the call of the application is similar easy compared with the native executable, while there is no native stuff inside and therefore it is easy exchangeable between operating systems.

Using the bnd-export-maven-plugin you can achieve the same result even with a PDE-Tycho based build. But of course you need to prepare things to make it work.

The first thing to know is that the bnd-export-maven-plugin needs a bndrun file as input. So now create a file headless.bndrun in org.fipro.headless.product project that looks similar to this:

-runee: JavaSE-1.8
-runfw: org.eclipse.osgi
-runsystemcapabilities: ${native_capability}

-resolve.effective: active;skip:="osgi.service"

-runrequires: \
osgi.identity;filter:='(osgi.identity=org.fipro.headless.app)'

-runbundles: \
org.fipro.inverter.api,\
org.fipro.inverter.command,\
org.fipro.inverter.provider,\
org.fipro.headless.app,\
org.apache.felix.gogo.command,\
org.apache.felix.gogo.runtime,\
org.apache.felix.gogo.shell,\
org.eclipse.equinox.console,\
org.eclipse.osgi.services,\
org.eclipse.osgi.util,\
org.apache.felix.scr

-runproperties: \
osgi.console=

• As we want our Eclipse Equinox based application to be bundled as a single executable jar, we specify Equinox as our OSGi framework via -runfw: org.eclipse.osgi.
• Via -runbundles we specify the bundles that should be added to the runtime.
• The settings below -runproperties are needed to handle the Equinox OSGi console correctly.

Unfortunately there is no automatic way to transform a PDE product definition to a bndrun file, at least I am not aware of it. And yes there is some duplication involved here, but compared to the result it is acceptable IMHO. Anyhow, with some experience in scripting it should be easy to automatically create the bndrun file out of the product definition at build time.

Now enable the bnd-export-maven-plugin for the product build in the pom.xml of org.fipro.headless.product. Note that even with a pomless build it is possible to specify a specific pom.xml in a project if something additionally to the default build is needed (which is the case here).

<plugin>
  <groupId>biz.aQute.bnd</groupId>
  <artifactId>bnd-export-maven-plugin</artifactId>
  <version>4.3.1</version>
  <configuration>
    <failOnChanges>false</failOnChanges>
    <bndruns>
      <bndrun>headless.bndrun</bndrun>
    </bndruns>
    <bundles>
      <include>${project.build.directory}/repository/plugins/*</include>
    </bundles>
  </configuration>
  <executions>
    <execution>
      <goals>
        <goal>export</goal>
      </goals>
    </execution>
  </executions>
</plugin>

The bndruns configuration property points to the headless.bndrun we created before. In the bundles configuration property we point to the build result of the tycho-p2-repository-plugin to build up the implicit repository. This way we are sure that all required bundles are available without the need to specify any additional repository.

After a new build you will find the file headless.jar in org.fipro.headless.product/target. You can start the command line application via

java -jar headless.jar

You will notice that the OSGi console is started, anyhow which parameters are added to the command line. And all the command line parameters are not evaluated, because not the Equinox launcher started the application. Instead the bnd launcher started it. Therefore the EnvironmentInfo is not initialized correctly.

Unfortunately Equinox will anyhow publish the EnvironmentInfo as a service even if it is not initialized. Therefore the EquinoxStarter will be satisfied and activated. But we will get a NullPointerException (that is silently catched) when it is tried to access the framework and/or non-framework args. For good coding standards the EquinoxStarter needs to check if EnvironmentInfo is correctly initialized, otherwise it should do nothing. The code could look similar to this snippet:

@Component(immediate = true)
public class EquinoxStarter {

  @Reference
  EnvironmentInfo environmentInfo;

  @Reference
  StringInverter inverter;

  @Activate
  void activate() {
    if (environmentInfo.getFrameworkArgs() != null
      && environmentInfo.getNonFrameworkArgs() != null) {

      // check if -console was provided as argument
      boolean isInteractive = Arrays
        .stream(environmentInfo.getFrameworkArgs())
        .anyMatch(arg -> "-console".equals(arg));
      // check if -console was provided as argument
      boolean showConsoleLog = Arrays
        .stream(environmentInfo.getFrameworkArgs())
        .anyMatch(arg -> "-consoleLog".equals(arg));

      for (String arg : this.environmentInfo.getNonFrameworkArgs()) {
        System.out.println(inverter.invert(arg));
      }

      // If the -consoleLog parameter is used, a separate shell is opened. 
      // To avoid that it is closed immediately a simple input is requested to
      // close, so a user can inspect the outputs.
      if (showConsoleLog) {
        System.out.println();
        System.out.println("***** Press Enter to exit *****");
        // just wait for a Enter
        try (BufferedReader reader = new BufferedReader(new InputStreamReader(System.in))) {
          reader.readLine();
        } catch (IOException e) {
          e.printStackTrace();
        }
      }

      if (!isInteractive) {
        // shutdown the application if no console was opened
        // only needed if osgi.noShutdown=true is configured
        System.exit(0);
      }
    }
  }
}

This way we avoid that the EquinoxStarter is executing any code. So despite component instance creation and destruction, nothing happens.

To handle launching via bnd launcher, we need another starter. We create a new immediate component named BndStarter.

@Component(immediate = true)
public class BndStarter {
    ...
}

The bnd launcher provides the command line parameters in a different way. Instead of EnvironmentInfo you need to get the aQute.launcher.Launcher injected with its service properties. Inside the service properties map, there is an entry for launcher.arguments whose value is a String[]. To avoid the dependency to aQute classes in our code, we reference Object and use a target filter for launcher.arguments which works fine as Launcher is published also as Object to the ServiceRegistry.

String[] launcherArgs;

@Reference(target = "(launcher.arguments=*)")
void setLauncherArguments(Object object, Map<String, Object> map) {
    this.launcherArgs = (String[]) map.get("launcher.arguments");
}

Although not necessary, we add some code to align the behavior when started via bnd launcher with the behavior when started with the Equinox launcher. That means we check for the -console parameter and stop the application if that parameter is missing. The check for -consoleLog would also not be needed, as the bnd launcher stays in the same command shell like eclipsec.exe, but for processing we also remove it. Just in case someone tries it out.

The complete code of BndStarter would then look like this:

@Component(immediate = true)
public class BndStarter {

  String[] launcherArgs;

  @Reference(target = "(launcher.arguments=*)")
  void setLauncherArguments(Object object, Map<String, Object> map) {
    this.launcherArgs = (String[]) map.get("launcher.arguments");
  }

  @Reference
  StringInverter inverter;

  @Activate
  void activate() {
    boolean isInteractive = Arrays
      .stream(launcherArgs)
      .anyMatch(arg -> "-console".equals(arg));

    // clear launcher arguments from possible framework parameter
    String[] args = Arrays
      .stream(launcherArgs)
      .filter(arg -> !"-console".equals(arg) && !"-consoleLog".equals(arg))
      .toArray(String[]::new);

    for (String arg : args) {
      System.out.println(inverter.invert(arg));
    }

    if (!isInteractive) {
      // shutdown the application if no console was opened
      // only needed if osgi.noShutdown=true is configured
      System.exit(0);
    }
  }
}

After building again, the application will directly close without the -console parameter. And if -console is used, the OSGi console stays open.

The above handling was simply done to have something similar to the Eclipse product build. As the Equinox launcher does not automatically start all bundles the -console parameter triggers a process to start the necessary Gogo Shell bundles. The bnd launcher on the other hand always starts all installed bundles. The OSGi console always comes up and can be seen in the command shell even before the BndStarter kills it. If that behavior does no satisfy your needs, you could also easily build two application variants: one with a console and one without. You simply need to create another bndrun file that does not contain the console bundles and no console configuration properties.

-runee: JavaSE-1.8
-runfw: org.eclipse.osgi
-runsystemcapabilities: ${native_capability}

-resolve.effective: active;skip:="osgi.service"

-runrequires: \
    osgi.identity;filter:='(osgi.identity=org.fipro.headless.app)'

-runbundles: \
    org.fipro.inverter.api,\
    org.fipro.inverter.provider,\
    org.fipro.headless.app,\
    org.eclipse.osgi.services,\
    org.eclipse.osgi.util,\
    org.apache.felix.scr

If you add that additional bndrun file to the bndruns section of the bnd-export-maven-plugin the build will create two exports.

<plugin>
  <groupId>biz.aQute.bnd</groupId>
  <artifactId>bnd-export-maven-plugin</artifactId>
  <version>4.3.1</version>
  <configuration>
    <failOnChanges>false</failOnChanges>
    <bndruns>
      <bndrun>headless.bndrun</bndrun>
      <bndrun>headless_console.bndrun</bndrun> 
    </bndruns>
    <bundles>
      <include>target/repository/plugins/*</include>
    </bundles>
  </configuration>
  <executions>
    <execution>
      <goals>
        <goal>export</goal>
      </goals>
    </execution>
  </executions>
</plugin>

To check if the application should be stopped or not, you then need to check for the system property osgi.console.

boolean hasConsole = System.getProperty("osgi.console") != null;

If a console is configured to not stop the application. If there is no configuration for osgi.console call System.exit(0).

This tutorial showed a pretty simple example to explain the basic concepts on how to build a command line application from an Eclipse project. A real-world example can be seen in the APP4MC Model Migration addon, where the above approach is used to create a standalone model migration command line tool. This tool can be used in other environments like in build servers for example, while the integration in the Eclipse IDE remains in the same project structure.

The sources of this tutorial are available on GitHub.

If you are interested in finding out more about the Maven plugins from bnd you might want to watch this talk from EclipseCon Europe 2019. As you can see they are helpful in several situations when building OSGi applications.

Update: configurable console with bnd launcher

I tried to make the executable jar behavior similar to the Equinox one. That means, I wanted to create an application where I am able to configure via command line parameter if the console should be activated or not. Achieving this took me quite a while, as I needed to find out what causes the console to start with Equinox or not. The important thing is that the property osgi.console needs to be set to an empty String. The value is actually the port to connect to, and with that value set to an empty String, the current shell is used. In the bndrun files this property is set via -runproperties. If you remove it from the bndrun file, the console actually never starts, even if passed as system property on the command line.

Section 19.4.6 in Launching | bnd explains why. It simply says that you are able to override a launcher property via system property. But you can not add a launcher property via system property. Knowing this I solved the issue by setting the osgi.console property to an invalid value in the -runproperties section.

-runproperties: \
    osgi.console=xxx

This way the application can be started with or without a console, dependent on whether osgi.console is provided as system parameter via command line or not.

Of course the check for the -console parameter should be removed from the BndStarter to avoid that users need to provide both arguments to open a console!

I added the headless_configurable.bndrun file to the repository to show this:

Launch without console:

java -jar headless_configurable.jar Test

Launch with console:

java -jar -Dosgi.console= headless_configurable.jar

Update: bnd-indexer-maven-plugin

I got a pull request that showed an interesting extension to my approach. It uses the bnd-indexer-maven-plugin to create an index that can then be used in the bndrun files to make it editable with bndtools.

<plugin>
  <groupId>biz.aQute.bnd</groupId>
  <artifactId>bnd-indexer-maven-plugin</artifactId>
  <version>4.3.1</version>
  <configuration>
    <inputDir>${project.build.directory}/repository/plugins/</inputDir>
  </configuration>
  <executions>
    <execution>
      <phase>package</phase>
      <id>index</id>
      <goals>
        <goal>local-index</goal>
      </goals>
    </execution>
  </executions>
</plugin>

To make use of this you first need to execute the build without the bnd-export-maven-plugin so the index is created out of the product build. After that you can create or edit a bndrun file by adding these lines on top:

index: target/index.xml;name="org.fipro.headless.product"

-standalone: ${index}

I am personally not a big fan of such dependencies in the build timeline. But it is surely helpful for creating the bndrun file.