Vertx

Embed Vert.x eventbus into your Spring Application with ease.

So, Vert.x 2.0+ now makes it so easy to embed Vert.x into any application. In our case, we wanted to embed Vert.x in a Spring MVC Rest Web Application, so that we can send messages from the Web Application to the eventbus to be picked up by our Vert.x application. All these messages are to be sent from the server side, which is why we did not simply choose to use Vert.x in our client side JavaScript, which is also very easy to do. We actually wanted the sending calls to be done by the back end Services, not even the Controllers of the web app. You know server side services that are re-usable. 😉

Anyway, using Vert.x documentation (Yes, you can learn this from the docs too), I found that it was extremely easy.

Biggest challenge was making sure the correct Vert.x jars are in your classpath of your web application. If you are using Maven, we simply added


        <!-- Vert.x jars to be included-->
        <dependency>
          <groupId>io.vertx</groupId>
          <artifactId>vertx-core</artifactId>
          <version>${vertx.version}</version>
          <exclusions>
              <exclusion>
                  <groupId>log4j</groupId>
                  <artifactId>log4j</artifactId>
              </exclusion>
          </exclusions>
        </dependency>
        <dependency>
          <groupId>io.vertx</groupId>
          <artifactId>vertx-platform</artifactId>
          <version>${vertx.version}</version>
        </dependency>

Those dependencies will pull in other dependencies too. Here is the mvn dependency:tree output for those two dependencies

+- io.vertx:vertx-core:jar:2.0.0-final:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.2.2:compile
[INFO] | | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.2.2:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.2.2:compile
[INFO] | +- com.hazelcast:hazelcast:jar:2.5:provided (version managed from 2.6; scope managed from compile)
[INFO] | \- io.netty:netty-all:jar:4.0.2.Final:compile
[INFO] \- io.vertx:vertx-platform:jar:2.0.0-final:compile

Then for getting the EventBus in our code, we wanted to be able to simply @Autowire into any class that we wanted. So we used an @Configuration class that instantiates Vertx and gets the EventBus from it as an @Bean.

Simple as


@Configuration
@PropertySource(value="classpath:vertx.properties")
public class OurAppConfiguration.class {

  private static final String VERTX_HOSTNAME = "vertx.hostname";

  @Autowired
  Environment environment;

  @Bean
  public EventBus eventBus() {
    Vertx vertx = VertxFactory.newVertx(environment.getProperty(VERTX_HOSTNAME));
    return vertx.eventBus();
  }
}

So as you can see we have a vertx.properties file with one property in it. You can, of course, add more properties, like port number and such. But just remember to add that to the call to VertxFactory.newVertx. As we do in the first line of the @Bean method. And just return vertx.eventBus(). Easy as pie.

Remember if you want this EventBus to work with another Vert.x application running somewhere else within your network, to run that one with -cluster, so that clustering works (If you need clustering)

To test this out, we wrote a simple JUnit Test with Spring integrated into it. Here is the code.


// YOU MUST BE RUNNING other Vert.x application with -cluster to run this test.
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes=[OurAppConfiguration.class])
class TestVertxEventBus {

  @Autowired
  EventBus eventBus

  @Test
  public void testSendingMessageOnEventBusToOtherVertxApplicationHelloWorld() {
    CountDownLatch startSignal = new CountDownLatch(1);
    eventBus.send("test.address", sendMessage, new Handler<Message<String>>(){
        void handle(Message<String> message) {
          assertNotNull(message)
          String response = message.body()
          assertEquals("Well Hello back", response)
          startSignal.countDown()
        }
      }
    )
    startSignal.await()
  }
}

I had to alter some of the code because it was specific to our application, which I am not at liberty to post. But you should be able to make an extremely simple Verticle that listens to the eventBus at “test.address” and responds back. Simple echo, or request-reply works. Vert.x has that in the examples right up front.

So I am not going to post one here. I’ll just say, if you don’t know how to write a real simple Verticle, then don’t try to embed Vert.x in another application just yet. While it is really easy to do, it is still and advanced Vert.x topic

As always, don’t even try to post spam that tries to look like a real response. It is too easy to see and they will never get onto this page.

Converting your Vert.x 1.x module to Vert.x 2.x

In this post, I am going to list the number of things we needed to do in order to upgrade our application/modules to Vert.x 2.0.

It is more a list than prose, because I need to create this post as I go along the ride. Because I am sure I will forget something that I did, if I do this post-mortem (Is that the correct way to use that term, or is it supposed to be something else.

1) Make sure all your modules match the module naming standard. Vert.x 2.0 will not run your module unless it matches. It has code to catch it immediately.
2) You probably want to upgrade your build to Gradle, since the Gradle template starts things off nicely. This can be very difficult if you are currently using Maven and have lots of dependencies. You will need to add all those dependencies to the gradle.build file. Which is why at this point we haven’t done it yet, I converted my Maven build to a better Maven pom based on Vert.x Maven Archetype. (see paragraph after next)

Another Gradle recommendation is move all the versioning of dependencies into the gradle.properties file.

Option – If you are converting your application that is currently in Maven, and you want to jump to Gradle. First recommendation, don’t. At least not at first, get your application upgraded to Vert.x 2.0, which requires changes to your Maven pom files (Could be lots of different one). Then once you have it working, then look at converting your maven build to Gradle.

For all the build stuff, the approach I took to change our maven pom file was to create a test Vert.x 2.0 project from the new Vert.x maven archetype. Then look at the pom file it creates and basically copy and paste things. Now each project that you have to update, might need different things changed, removed or added. However, really try to get your pom file to match the pom file that that archetype created.

With Vert.x 2.x you will also really want to take advantage of the testtools jar file as it has now become really easy to write Vert.x eventbus integration tests with TestVerticle. I might create a blog post just on that subject later. One that includes writing the integration tests as well as how you can use a TestVerticle to fire off your Vert.x appication within an IDE.

Overall, I would say those are the two main factors to watch out for. 1) naming of your modules and 2) Bigger issue to update your build files. As far as any code changes, there really shouldn’t be any.

Configuring Maven to build your vertx module

How to create a vertx mod from your Maven project.

So you want to package your vertx application into a module and then make it available in some vertx repository to automatically download. How do you do that? If you are using Gradle for your build, check out all the mods that vertx has already created to examples, then copy and paste and change to your project.

But there aren’t any out there with maven as a build. Well, Brian Lalor does have a maven build for his amqp module, and that is exactly where I stole this information from. Now in his project, as of today, it creates a tar.gz file, I want to create a zip, that I can then copy to my mod repository and rename to mod.zip.

So what are we going to do? Well, for our project I decided to use the maven assembly plugin, maven-assembly-plugin. And create an xml file that declared that I wanted a zip file, what jar dependencies to copy over into a lib directory and where my mod.json file was located, as well as any other scripts or code that wasn’t already being packaged into a jar file by the build. Then in the maven-assembly-plugin, point it to the xml file with that configuration.

First lets look at the maven assembly plugin entry in our pom file.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-assembly-plugin</artifactId>
    <version>2.3</version>

    <configuration>
        <descriptors>
            <descriptor>src/main/assembly/mod.xml</descriptor>
        </descriptors>
    </configuration>

    <executions>
        <execution>
            <id>assemble</id>
            <phase>package</phase>
            <goals>
                <goal>single</goal>
            </goals>
        </execution>
    </executions>
</plugin>

This declares the plugin, in the configuration section points to where I will have my xml file, and finally the execution, so that when I run package or beyond that the assembly is done.

So that just leaves us with this mod.xml file thingy. let’s take a look at it. (I create a property in the pom.xml file called

<module-directory>${project.groupId}.${project.artifactId}-v${project.version}</module-directory>

This way can use the property to tell the mod.xml locations to place my vertx module code and stuff.

Now the mod.xml file

<?xml version="1.0" encoding="UTF-8"?>
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="
              http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd
          ">

    <id>mod</id>
    <formats>
        <format>zip</format>
    </formats>

    <baseDirectory>mods</baseDirectory>

    <dependencySets>
        <dependencySet>
            <outputDirectory>${module-directory}/lib</outputDirectory>
        </dependencySet>
    </dependencySets>

    <fileSets>
        <fileSet>
            <outputDirectory></outputDirectory>
            <directory>src/main/resources</directory>
            <includes>
                <include>amqp-busmod#1.2.0-SNAPSHOT/**</include>
            </includes>
        </fileSet>
        <fileSet>
            <!-- empty outputDirectory puts mod.json at the root of the structure -->
            <outputDirectory>${module-directory}</outputDirectory>
            <directory>src/main/resources</directory>
            <includes>
                <include>mod.json</include>
            </includes>
        </fileSet>
        <fileSet>
            <outputDirectory>${module-directory}</outputDirectory>
            <directory>scripts</directory>
            <includes>
                <include>**</include>
            </includes>
        </fileSet>
    </fileSets>

</assembly>

WHOA!!!! That’s a lot. But most is boilerplate.

First the id will be appended to the name of the zip file it creates. Now it creates a zip file because of the formats/format section. That is simple.

Now in vertx mods are put into a subdirectory called mods. So in the baseDirectory tag we tell it to put all our files down one level in the zip file, so that the root level in the zip is “mods”

The dependencySets section is just telling where the jar files should be copied to. In this case the module’s directory, but in a lib subdirectory of it. Basically mods/{module-directory}/lib

And the last part is the fun part. This is where we tell the assembly which of our files we want in the module directory, or if we are using other modules where to find those modules and copy those directly into the mods directory. This is the fileSets section

So in my mod.xml I am using Brian’s amqp module (the same one I stole from) and I downloaded it locally and put it in my src/main/resources directory. I am actually also using the MongoDBPersistor module, but I am going to get that from the vertx central repository when I actually run my module. So the fileSet for that has an empty outputDirectory, which means it will be copied into the baseDirectory setting which is set to mods directory.

The second fileSet is for my mod.json, which is required for a vertx module and tells vertx the main script/class and maybe some other configuration.

And finally the last fileSet is my actual verticle code. I am using Groovy scripts, so I created a scripts directory off the root of my project’s directory and placed my Groovy scripts there, in my case there are actually 5 Groovy script files; 1 verticle, 2 worker verticle, 1 main app script to deployVerticle, deployMod etc, and one Groovy config file. The last is not necessary, but will also be part of another blog post for vertx using environment configurations. But I haven’t gotten that to work yet. ANYWAY, since I have a few files in this scripts directory I just use the ant style “**” wildcards to say take everything in that scripts directory and also copy those to my module’s directory. So in the mods/{module-directory/ directory.

Now when I do mvn package, I look in the target directory when it is done and I see a zip file with my module name on it with “-mod.zip” at the end.

So what I usually do next is unzip that file directly in the target directory so I can run my module directly from the target directory and test away.

Due to complete ass wipe spammers, I have to turn off comments and trackbacks.

Vertx with Groovy config files and environments

(CAVEAT. I don’t think I ever tested this, so there might be one or two tweaks that need to be done)

So vertx has ways to write configs for your verticles.  They are in code, in a .json file with -conf on the command line, in a .js file with load() in js. But nothing built in with Groovy.

You can put them in your code and load them programatically. However, with that route, your config is in your code and changes to the config would require changing code. So then you would want to externalize your config. And vertx allows you to tell it what config file to load with the -conf command line argument.

If you use this approach, then where do you put your config files. In this case, they would have to be in the directory that you are running your verticle. This is fine if you are in the same directory as the verticles, which means a simple verticle not in a module. So now you want to package your verticle(s) into a module. This now means that your config files are in the directory that you are running from but not with the verticle in the module itself. This is also fine and what you typically do when you are using a module say from the vertx mods repository.

So lets now say you want to create configs for different environments and store that config in a .groovy file and in the same directory as your module. And you also have more than one verticle in your module. How do you go about that? This is what this blog post is about. How to do just that. This will still require a .json config file in the directory you run from, but that .json can be as simple as stating what environment you want to run with. So very simple

config.json contains

{ "env" :"prod"}

or
{"env" :"dev"}

So when you want to change your environment you are using you just have to change that one simple property in the config.json file.

Now lets look at how we are going to write our actual config for our verticles using a Groovy script.

I have a Config.groovy file that looks like this

environments {
    dev {
        myVerticleOneConfig: {
            numberOfInstances: 10
        }
        myVerticleTwoConfig: {
            numberOfInstances: 10
        }
        dataIntegrationWorkerConfig: {
            uri: "amqp://localhost:5672"
            defaultContentType: "application/json"
            numberOfInstances: 10
        }
        mongoDBPersisterWorkerConfig: {
            host: "localhost"
            port: 27017
            db_name: "addressBook"
            numberOfInstances: 10
        }
    }
    test {
        myVerticleOneConfig: {
            numberOfInstances: 50
        }
        myVerticleTwoConfig: {
            numberOfInstances: 50
        }
        dataIntegrationWorkerConfig: {
            uri: "amqp://192.168.0.10:5672"
            defaultContentType: "application/json"
            numberOfInstances: 50
        }
        mongoDBPersisterWorkerConfig: {
            host: "192.168.0.11"
            port: 27017
            db_name: "addressBook"
            numberOfInstances: 20
        }
    }
    prod {
        myVerticleOneConfig: {
            numberOfInstances: 100
        }
        myVerticleTwoConfig: {
            numberOfInstances: 100
        }
        dataIntegrationWorkerConfig: {
            uri: "amqp:/63.18.0.10:5672"
            defaultContentType: "application/json"
            numberOfInstances: 100
        }
        mongoDBPersisterWorkerConfig: {
            host: "63.18.0.11"
            port: 27017
            db_name: "addressBook"
            numberOfInstances: 30
        }
    }
}

Now I want to use these configs in my vertx module/application. One you have more than one verticle to deploy in your app with other mods and verticles and worker verticles, I highly recommend creating a vertx module for your app and to create a main app that deploys your verticles. like

MyApp.groovy file

// Config variable definition will be going here

container.deployVerticle("VerticleOne.groovy", config.verticleOne.numberOfInstances)
container.deployWorkerVerticle("VerticleTwoWorker.groovy", config.verticleTwo.numberOfInstances)
container.deployModule(
        "amqp-busmod#1.2.0-SNAPSHOT",
        config.dataIntegrationWorkerConfig,
        config.dataIntegrationWorkerConfig.numberOfInstances
)

container.deployModule(
        "vertx.mongo-persistor-v1.1",
        config.mongoDBPersisterWorkerConfig,
        config.mongoDBPersisterWorkerConfig.numberOfInstances
)

So in this we deploy our verticles and worker verticles and repo mods passing in configuration and number of instances using our external configuration which we will assign to a variable called config.

So the next part is to define that variable “config”. With Groovy’s ConfigSlurper we can load in the Config.groovy file with the following line of code

def config = new ConfigSlurper(container.config.env).parse(new File('Config.groovy').toURL())

That line goes to the top of the MyApp.groovy file.

Notice the constructor of ConfigSluper I am passing in container.config.env . This will come from reading in the config.json file being passed to -conf at the command line. If the env property in config.json is equal to “dev” then the dev section in the Config.groovy will fill all the values under the “config” variable and you are all set.

Now when I run my module it will be with the following command line

vertx runmod com.pwp.mymodule-v1.0.0 -conf config.json