Chapter 35. Getting Started as a Developer

This chapter describes the basic steps to get started as a developer for the OpenEngSB project.

35.1. Getting comfortable with the infrastructure

As any open source project the OpenEngSB development depends on a wide range of different infrastructural and communication methods to get things done. The following sub-chapters describe the different tools, their location and usage in the OpenEngSB development process.

35.1.1. Mailing Lists

The most important communication medium for the OpenEngSB development team is email. Mostly all of the vital design, architectural and infrastructural decisions are discussed on the OpenEngSB developer list. Therefore, the first step to get involved into the development of the OpenEngSB is to register to the Google Groups OpenEngSB Developer Mailing List and say hello world.

While notifications from the Hudson Build Server, about code commits and Jira issues are vital for the OpenEngSB core developer, they may not be as interesting for you. If you get annoyed by the automatically generated notification mails ignore all mails from openengsb@gmail.com and noreply@github.com to openengsb-dev@googlegroups.com. Please remember it is important to configure both, to and from in your filter. Both addresses will also send notifications directly to you which are important and should not be ignored!

35.1.2. Jira Issue Tracker

All issues are stored within a Jira instance reachable at issues.openengsb.org. Please use the issue tracker to keep track of all bugs, ideas and new features you're currently working or of which you think they might be interesting.

35.1.3. Code Repository

As for any open source project the source code is public available. We've chosen Github for this task. The project is available at github.com/openengsb/openengsb.

As explained later within this document Github is not only used to store our code, but also for collaboration, code review and patch-tracking.

35.1.4. Maven Repository

The OpenEngSB is available at Maven Central. We still have our own Maven repository at maven.openengsb.org/ and snapshots are available via the sonatype Maven repository at http://oss.sonatype.org.

35.1.5. Build Server

The master and integration branch of the OpenEngSB repository are watched and built by a Hudson build server instance available at build.openengsb.org. Notifications about failures are directly sent to the OpenEngSB developer list.

35.2. Prerequisites

First of all the latest JDK has to be installed on the system and the JAVA_HOME variable has to be set accordingly. All further steps are described in the subsections of this chapter.

35.2.1. Installing Git

It is assumed that Git is installed. For Linux your distribution provides already a package for git. Please use the package manager of your distribution (apt, yum, pacman, ...) to install it. For MAC binaries are available at git-scm.com. For MS users cygwin or msysgit. After installing, set at least the following variables:


        git config --global user.name "Firstname Lastname"
        git config --global user.email user@example.com
        git config --global core.autcrlf input
      

35.2.2. Installing Maven

Finally download Apache Maven3 and unpack it. Add the path of the maven binary to your PATH variable. Furthermore you should set the MAVEN_OPTS environment variable to allow Maven to use more RAM. If you don't you'll get Out Of Memory errors.


          export PATH=$PATH:/path/to/maven/bin
          export MAVEN_OPTS='-Xmx1024M -XX:MaxPermSize=512m'
        

Add these commands to ~/.bashrc to make the settings permanent.

35.3. Starting OpenEngSB

The next step is to get the OpenEngSB source by checking out the current master using git:


      git clone git://github.com/openengsb/openengsb.git
    

Connector, domains and additional various features are included into the OpenEngSB as submodules. To include them into the build and development cycle execute ./etc/scripts/submodulePush.sh if you're a commiter and ./etc/scripts/submodulePull.sh otherwise.

Now start the OpenEngSB by executing


      mvn clean install openengsb:provision
    

This command builds, tests and runs the OpenEngSB right from your command-line. Executing the following command will shutdown it again:


      shutdown
    

35.4. Using Eclipse

Eclipse had been chosen by the OpenEngSB team as the main development environment. After checkout the code the following command creates the required Eclipse project files:


      mvn install
      mvn openengsb:eclipse
    

Start Eclipse and select any workspace. The folder eclipse-workspace is ignored in the OpenEngSB project structure for this purpose. But you can choose any other directory if you prefer. At the preference page go to Java/Build Path/Classpath Variables and create a new M2_REPO pointing to ~/.m2/repository. Now use File, Import..., Existing Projects into Workspace. As the root directory select the root of the OpenEngSB source. Eclipse will list several projects and for now it's best to import them all by clicking Finish.

At openengsb/etc/eclipse/ eclipse configuration files for formatting can found. Checkstyle configuration files are part of the openengsb-maven-plugin (Section 35.7.1, “openengsb-maven-plugin”) and can be found here. The mvn openengsb:eclipse goal configures your eclipse project to download and use the provided checkstyle configuration file, so no manual steps are necessary here (however the formatter still needs to and should be configured manually).

35.5. Using Other IDEs than Eclipse

Basically, the OpenEngSB is developed in plain Java, which means any other IDE than Eclipse can be used too. While there are tools for most IDEs to use Checkstyle, but non of it supports the formatting file of the OpenEngSB. Please use Checkstyle, which automatically validates the eclipse formatting rules too.

35.6. Git Documentation

35.6.1. Usage

First of all this chapter explains only the very basics of Git and only that parts directly relevant for the development of the OpenEngSB project, but not the entire idea and possibilities of Git. Please read some tutorials first to get how to work with Git and see this chapter more as an summary! You may also take a look at the Git Documentation Page and the Pro Git Book.

35.6.2. Github

OpenEngSB is developed at github.com. Please create an account there and explore its features. Specify your real name in the admin tab and add a picture. This makes it easier to associate your commits to you.

35.6.3. Starting up and configure

Before starting to work with Git some settings should be applied to Git. Therefore simply execute the following commands.

git config --global user.name "Firstname Lastname"
git config --global user.email user@example.com
git config --global color.ui "auto"
git config --global pack.threads "0"
git config --global diff.renamelimit "0"
git config --global core.autocrlf "input"

Additionally execute the special settings for github as could be found on github in the "Account Settings" tab is a point "Global git config information". Please use the two git commands described there

git config --global github.user username
git config --global github.token token

If you don't already have an SSH key you can create one by executing ssh-keygen Simply answer all questions from the application with "enter" without enter any values. Afterwards the content of the id_rsa.pub file from your ~/.ssh/ directory should be submitted to github (Account Settings/SSH public keys).

You may also want to setup the provided git-hooks. Hooks are scripts that automate some small tasks in the git-workflow. To enable them they have to be located in the .git/hooks directory. You can achieve this by copying the scripts located in the etc/git-hooks directory to .git/hooks

35.6.4. Contributor Workflow

Contributor are all developer who like to contribute to the OpenEngSB project, but not have commit rights to openengsb/openengsb.

Please keep in mind, that this manual is NOT a Git tutorial. Github itself, e.g. provides a great help at help.github.com. All base concepts such as forking, pull-requests, ...

Please start by choose or create a new issue. Now create a new fork of the OpenEngSB at Github (if you've not done already so; otherwise this is explained here). Clone your fork, but also add the original openengsb repository as remote repository. This is also explained here. In difference to the Github tutorial please do not commit to the master, but rather create a new branch named OPENENGSB-ISSUE_NUMBER_YOURE_WORKING_ON. Optionally append /DESCRIPTION (e.g. OPENENGSB-586/mvn-eclipse-download-fix).

git checkout -b OPENENGSB-ISSUE origin/BRANCH

BRANCH is the point where you like to start your work. If you like to contribute to the head this will be typically integration, but could also be a commit or a complete different branch. This is the OpenEngSB schema for naming branches and we'll really appreciate if you work according to it. Please notice that each commit should include [OPENENGSB-ISSUEID] at the beginning of its message.

Now hack, commit and push as you like. If you think you're finished invoke mvn openengsb:prePush (or use etc/scripts/pre-push.sh) to validate your code, tests, licenses and so on. If everything works without errors create a Github pull request on Github, between the master or integration branch (depending on where you've created your branch on) and your branch. This process is also explained at help.github.com (here). In addition it will help if you add the link to the pull request to the issue you're working on. A commiter will tend as fast as possible to your request and give feedback or directly merge your commit into the integration/master branch.

35.6.5. Commiter Workflow

The only difference between a commiter and a contributor is that he has to watch and merge branches of contributors. If a commiter is happy with the work of a contributor. Comments and other discussions should be done on the mailing list and/or via the Github review system and pull requests.

In addition commiters typically do not create forks but rather create their branches directly in the OpenEngSB repository. This is done because the repository is covered by the OpenEngSB build server and in addition keeps everything closer together.

35.6.6. Additional Rules

  1. (Contributor/Committer) All development is done in branches (also of the core developers) One exception to this rule exists: Small fixes and maintenance work which is NOT related to a new feature and does not exceed 2 commits should be cherry-picked into the master directly.

  2. (Contributor/Committer) Rebase is not dead (although we use merges). Never ever commit local merges. You still should develop in local dev branches and rebasing them with the upstream branches. Only if nobody else has access to your fork you can be sure that nobody changed it!

  3. (Committer) If merging branches from forked repositories ALWAYS use the --no-ff option for merges; this will always create a merge node (even if a fast-forward merge is possible). This is required to create a clear and consistent history!

  4. Avoid backward merges from the master and keep feature branches small! This does not mean that backward merges from master are forbidden. But they should not be done too often, since they create a history not easy to read. Please use the method described on this page (with --no-ff --no-commit) to reduce the number of merge nodes.

  5. Use meaningful feature branch names. Using the merge history in the master you can easily follow the development of features. But this requires (maybe long) good names! In addition, always start with OPENENGSB-NUMBER of the issue you're working on. Try to always do work based on issues. If no issue covers what you're doing create one.

35.7. Useful Tools

35.7.1. openengsb-maven-plugin

The openengsb-maven-plugin is a plugin for Apache Maven, intended to simplify various activities (creating domains or connectors, building a release artifact of the whole project etc.) when developing based on the OpenEngSB.

35.7.1.1. Configuring the openengsb-maven-plugin for your project

To use the openengsb-maven-plugin in your project add the following configuration to your project's pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  ...
  <build>
    <plugins>
      <plugin>
        <groupId>org.openengsb</groupId>
        <artifactId>openengsb-maven-plugin</artifactId>
        <version>${openengsb.maven.plugin.version}</version>
      </plugin>
    </plugins>
  </build>
  ...
</project>       
        

The plugin can now be invoked via mvn openengsb:<goal>

35.7.1.2. Purpose of the openengsb-maven-plugin

TBD

35.7.1.3. Changing the default configuration of the mojos

TBD

35.7.1.4. Available Goals

assemble or etc/scripts/assemble.sh

Installs the OpenEngSB and skips tests. Furthermore a nightly profile is activated if available in your poms.

eclipse or etc/scripts/eclipse.sh

Generates eclipse configuration file for the module where it is invoked from and all submodules. The generated eclipse projects are also configured to use the checkstyle rules shipped with the plugin (see checkstyle mojo).

checkstyle

Performs a Checkstyle check of the project. The checkstyle configuration file which is used for the check can be found here. We ship this configuration file with the plugin (and changes need to be done there) because we think it may be useful for other OpenEngSB related projects. Setting up eclipse projects with configured checkstyle becomes very easy that way (simply mvn openengsb:eclipse).

genConnector or etc/scripts/gen-connector.sh (For additional info how to create a connector see Chapter 36, How To Create an Internal Connector)

Guides interactively through the creation of a connector and generates the artifact via the connector archetype.

genDomain or etc/scripts/gen-domain.sh (For additional info how to create a domain see Chapter 37, How To Create an Internal Domain)

Guides interactively through the creation of a domain and generates the artifact via the domain archetype.

licenseCheck or etc/scripts/license-check.sh

Performs a check if appropriate license headers are available in every source file. The licenseCheck mojo wraps the com.mycila.maven-license-plugin. A large part of the default behavior of this mojo can be changed in src/main/resources/license/licenseConfig.xml. See this site for available configuration options. The openengsb-maven-plugin needs to be reinstalled after changing its default behavior.

NOTE: pom.xml files are excluded from license check

Parameters:

  • additionalExcludes

    Defines path to a file where each line represents a pattern which files to exclude from license check or license format (additionally to the default excludes).

licenseFormat or etc/scripts/license-format.sh

Adds a license header to files where the license header is missing. Regarding the configuration for this mojo the same applies as for licenseCheck.

NOTE: pom.xml files are excluded from license format

Parameters:

prePush or etc/scripts/pre-push.sh

Builds and installs the openengsb, checks for license headers, performs a Checkstyle check and runs the integration tests.

Parameters:

provision or etc/scripts/run.sh / etc/scripts/quickrun.sh

Equivalent to execute karaf or karaf.bat per hand after build by mvn clean install in a (typically) assembly directory.

Parameters:

  • provisionPathUnix

    This setting should be done in the one of the assembly folders and have to point to the final directory where the karaf system, etc configs and so on consist.

  • provisionExecutionPathUnix

    The path to the executable in the unix archive file

  • additionalRequiredExecutionPathUnix

    Sometimes it's required that some executable files, provided in provisionExecutionPathUnix execute other files which have to made executable to work correctly on themselves. Those files should be specified here.

  • provisionPathWindows

    This setting should be done in the one of the assembly folders and have to point to the final directory where the karaf system, etc configs and so on consist.

  • provisionExecutionPathWindows

    The path to the executable in the windows archive file

  • additionalRequiredExecutionPathWindows

    Sometimes it's required that some executable files, provided in provisionExecutionPathWindows execute other files which have to made executable to work correctly on themselves. Those files should be specified here.

These parameters are typically configured in the pom of your assembly project (/assembly/pom.xml for the OpenEngSB)).

pushVersion or etc/scripts/push-version.sh

Updates the development version.

Parameters:

  • developmentVersion

    The new SNAPSHOT version.

releaseNightly or etc/scripts/release-nightly.sh

Mojo to perform nightly releases. This mojo activates the nightly profile in the project, where you can put your additional configuration for nightly releases (To see what these profiles can be necessary for please read the descript of the other release mojos).

release<XXX> (You can find a detailed description of the OpenEngSB release process in Chapter 39, Release and Release Process)

The release<XXX> mojos (except Nightly) wrap the maven-license-plugin, basically just invoking mvn release:prepare and then mvn release:perforn with some useful default configuration which can be reused for other projects related to the openengsb. These mojos perform a release and activate the <XXX> profile. These release profiles are important and are required to ..

  • .. select the appropriate passphrase for the maven release repository from your settings.xml. For additional information on this topic see Section 39.3, “Configure Maven”.

  • .. set links depending on the release type. For examples please see the profiles in the pom

  • .. configure distribution management of the project site, depending on the release type. For examples see profiles in docs/homepage/pom

Parameters:

  • connectionUrl

    URL to your SCM repository (e.g. scm:git:file://~/openengsb). During the release process changes (version updates, etc) are commited into your SCM.

Goals:

  • releaseFinal or etc/scripts/release-final.sh

    profile = final

  • releaseMilestone or etc/scripts/release-milestone.sh

    profile = milestone

  • releaseRC or etc/scripts/release-rc.sh

    profile = rc

  • releaseSupport or etc/scripts/release-support.sh

    profile = support