Our previous two posts have focused on the deployment infrastructure and continuous integration building blocks that we need to both enable our Kubernetes cluster and provide the foundation of our containerised deployments – continuously integrated builds.

One of the reasons we developed Connect is to take the learnings from our real-world implementations of microservice architectures and the problems we’ve tried to solve to create an ecosystem of modules that implement what we believe are best practices. These best practices might turn out to be wrong, or we might one day encounter a project that challenges some of our assumptions about how we think microservices should be built and used, and this is the reason for the modularity we have provided.

We’re now going to take a slight detour away from the operational aspects of our framework and this post will focus on addressing how to design for one of the properties a microservice application needs – namely, scalability. We’re not going to focus on how or why a certain architectural pattern or framework should or shouldn’t be used. Microservice architectures are new and fluid enough that there are still plenty of topics that are open for debate as engineers and organisations balance the agility that an increase in granularity and decomposition can bring against the inherent increase in complexity that a more distributed and disparate system brings.

Scalability in the context of this post means two things:

  1. The ability to replicate any given service and have it ready to accept work as quickly as possible.
  2. Any given service must be a ‘good neighbour’ since it is sharing resources that other services are using.

Monolithic applications don’t really have the need to start and stop as frequently as Microservices do and thus are generally not built to enable this paradigm. In a monolith, you might usually handle additional load by vertically scaling by adding or allocating additional CPU or RAM resources to your server. In a Microservice architecture, we handle additional load on the system by horizontally scaling (replicating) the components of the system that are subject to increased load so that we have more instances of the same service available to serve the additional load. What this means for our applications is that we continually need them to be able to start up and shut down cleanly and quickly.

Kubernetes allows us to apply very granular control to the resources our pods can request and use. On a recent Microservices (70+ services) project we encountered some problems with pod scaling whereby the startup load on a pod was very high as the application running on that pod scanned and enumerated its dependencies. This meant that the act of scaling out a pod caused such a spike in CPU resources that pods were unable to scale quickly enough to handle the incoming spikes in load that we experienced or simulated using load and soak testing. Our services were not exactly making good neighbours! This manifested itself in a couple of different ways – either our new pods would immediately exceed their CPU limits on startup and would trigger further horizontal scaling or they would be killed before they had a chance to start properly or they would cause cascading problems that led to node instability.

This post is going to focus on the components of the Connect framework that will help you build Java applications that start as quickly as possible so that they can be used with Kubernetes’ Horizontal Pod Autoscaler, or any autoscaling solution effectively. The primary component that enables this is the Connect Prescan Plugin. This component is a Maven plugin that you can use to perform classpath scanning at compilation rather than at runtime, while still encouraging modularity.

We try and design our Microservices so that they can be run with 400-500 millicores of CPU. A service with these resource restrictions should be able to perform a cold-start in less than 30 seconds, providing that it’s not required to do anything resource intensive like a database migration.

Let’s start by adding our prescan plugin to a project. We’re going to run through this example using one of our sample applications – a Slack sentiment analyser. Clone our sample apps repo as follows, or import it into an IDE of your choice:

git clone git@github.com:ClearPointNZ/connect-sample-apps.git; cd connect-sample-apps; cd slack-sentiment-analyser

This will put us into the slack-sentiment-analyser folder where we have a pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>cd.connect.samples.slackapp</groupId>
  <artifactId>slack-sentiment-analyser</artifactId>
  <packaging>jar</packaging>
  <version>1.1-SNAPSHOT</version>

  <properties>
    <activemq-version>5.15.0</activemq-version>
  </properties>

  <dependencies>
    <dependency>
      <groupId>cd.connect.samples.slackapp</groupId>
      <artifactId>slack-sentiment-analyser-api</artifactId>
      <version>1.1-SNAPSHOT</version>
    </dependency>

    <dependency>
      <groupId>cd.connect.composites.java</groupId>
      <artifactId>connect-composite-springwebapp</artifactId>
      <version>[1.1, 2)</version>
    </dependency>

    <dependency>
      <groupId>org.apache.activemq</groupId>
      <artifactId>activemq-all</artifactId>
      <version>${activemq-version}</version>
    </dependency>
  </dependencies>

  <build>
    <finalName>app</finalName>
    <plugins>
      <plugin>
        <groupId>io.repaint.maven</groupId>
        <artifactId>tiles-maven-plugin</artifactId>
        <version>2.10</version>
        <extensions>true</extensions>
        <configuration>
          <filtering>false</filtering>
          <tiles>
            <tile>cd.connect.tiles:tile-java:[1.1, 2)</tile>
          </tiles>
        </configuration>
      </plugin>
      <plugin>
        <groupId>cd.connect.common</groupId>
        <artifactId>connect-gen-code-scanner</artifactId>
        <version>1.1</version>
        <executions>
          <execution>
            <id>default</id>
            <goals>
              <goal>generate-sources</goal>
            </goals>
            <phase>generate-sources</phase>
            <configuration>
              <scanner>
                <packages>
                  <package>cd.connect.samples.slackapp-r=spring: dao, security</package>
                  <package>cd.connect.samples.slackapp.rest=spring/@Singleton</package>
                  <package>cd.connect.samples.slackapp.rest-r=jerseyuser</package>
                </packages>
                <templates>
                  <template>
                    <name>slackapp-config</name>
                    <template>/generator/common-spring.mustache</template>
                    <joinGroup>spring, servlet</joinGroup>
                    <className>cd.connect.samples.slackapp.SlackAppGenConfig</className>
                  </template>
                  <template>
                    <name>slackapp-data-resource</name>
                    <template>/generator/jersey.mustache</template>
                    <className>cd.connect.samples.slackapp.JerseyDataConfig</className>
                    <joinGroup>jersey=jerseyuser</joinGroup>
                    <context>
                      <baseUrl>/data/*</baseUrl>
                    </context>
                  </template>

                </templates>
              </scanner>
            </configuration>

          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

</project>

The code-scanner is a plugin so we’ve included it in our project by adding it to the <plugins> tag as a <plugin>. As this is a build plugin, it exists within the <build> tag. If you’re new to plugins within Maven, you might find the official documentation useful.

Let’s look in some more detail at the <scanner> tag where we configure our classpath scanner. Within this tag we have a number of <packages>. Each <package> enumerates the individual package that we want scanned.

The scanner plugin will only scan for source classes within the <package> specified. It won’t look for classes within those specific packages that are in jar dependencies as that is not a modular pattern.

<packages>
  <package>cd.connect.samples.slackapp-r=spring: dao, security</package>
  <package>cd.connect.samples.slackapp.rest=spring/@Singleton</package>
  <package>cd.connect.samples.slackapp.rest-r=jerseyuser</package>
</packages>

What the classpath scanner does is it looks through source classes that you put into the <package> tags and finds the interfaces that you specified that have been annotated with @. If you need a general overview or refresher on Java annotation, have a look through the documentation.

The generic pattern for each package is package[-modifiers]=group[,group…​]:[sub-package[,sub-package…​]][/[@Annotations,…​].

Our primary base package is cd.connect.samples.slackapp. We want the classpath scanner to grab us all the classes in the sub-packages dao and security and put them into the spring group. The modifier -r tells the scanner not to recurse when looking for packages, otherwise it will look recursively by default. dao and security are packages within the cd.connect.samples.slackapp path and spring is the group we want these packages to be put in.

We also want our singleton packages to be picked up. Our second package is cd.connect.samples.slackapp.rest. We want this to go into the spring group as well so we’ve followed the same pattern as the first package to achieve this. The /@Singleton part tells the scanner that we only want it to pick up classes that have the @Singleton annotation. You can restrict the classpath scanner to specifically annotated classes by using this pattern.

Our third package is cd.connect.samples.slackapp.rest but this time we’re not scanning recursively so we’ve used -r. The group we want to use is jerseyuser and there are no specific packages we’re looking for so cd.connect.samples.slackapp.rest-r=jerseyuser is all we need.

After we’ve enumerated all our packages, we then have a <templates> section.

<templates>
  <template>
    <name>slackapp-config</name>
    <template>/generator/common-spring.mustache</template>
    <joinGroup>spring, servlet</joinGroup>
    <className>cd.connect.samples.slackapp.SlackAppGenConfig</className>
  </template>
  <template>
    <name>slackapp-data-resource</name>
    <template>/generator/jersey.mustache</template>
    <className>cd.connect.samples.slackapp.JerseyDataConfig</className>
    <joinGroup>jersey=jerseyuser</joinGroup>
    <context>
      <baseUrl>/data/*</baseUrl>
    </context>
  </template>

</templates>

What this section does is it creates a list of Spring Beans and Jersey Resources that then need to be created and exposed by handing them off to mustache templates.

Taking the example of the first <template>, this will create a generated class named cd.connect.samples.slackapp.SlackAppGenConfig, using the <template> /generator/common-spring.mustache which looks as follows:

package {{packageName}};

import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Import;
{{#spring}}
  // plain resources
  {{#sortedTypes}}
import {{packageName}}.{{name}};
  {{/sortedTypes}}
{{/spring}}

@Configuration
@Import({ {{#spring}}{{#sortedTypes}}{{name}}.class{{^-last}}, {{/-last}}{{/sortedTypes}}{{/spring}} })
public class {{simpleName}} {
}

This is a simple template that just manages our Spring wiring but you can build templates of your own to do more complex things if your app requires it. For example, another template we’ve used will register Spring objects and wire up servlets from their declared structures.

We’ve provided some documentation with a couple of sample use cases in the Connect Prescan Plugin repo. Have a look at the readme for further instructions on how you can use the plugin to make your Java pods start like ⚡️.

Prev: Part 2: Bootstrapping Jenkins in a Kubernetes Cluster

Next: Test Automation of a Microservice using Cucumber, Java and OpenAPI

 


Leave a Reply

Your email address will not be published. Required fields are marked *