This week I have been trying out Cloud Foundry. Today while trying to set up monitoring through AppDynamics, the sample application that I was using started crashing because of a Java memory error. This blog post discusses how to prevent this from happening by configuring Java memory parameters used by the application.

This is what was in the Cloud Foundry logs.

May 20 18:14:51 CloudFoundry 6767b9fd-1874-43cb-a4f8-7470a17c90ae/[App/2]:  /bin/bash: line 31:    32 Killed                  ( SERVER_PORT=$PORT $PWD/.java-buildpack/open_jdk_jre/bin/java -cp $PWD/.:$PWD/.java-buildpack/spring_auto_reconfiguration/spring_auto_reconfiguration-1.4.0_RELEASE.jar -Djava.io.tmpdir=$TMPDIR -XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh -Xmx499200K -Xms499200K -XX:MaxPermSize=65M -XX:PermSize=65M -Xss1M -javaagent:$PWD/.java-buildpack/app_dynamics_agent/javaagent.jar -Dappdynamics.agent.applicationName='******' -Dappdynamics.agent.tierName='******' -Dappdynamics.agent.nodeName=$(expr "$VCAP_APPLICATION" : '.*instance_id[": ]*"\([a-z0-9]\+\)".*') -Dappdynamics.agent.accountAccessKey=****** -Dappdynamics.agent.accountName=******* -Dappdynamics.controller.hostName=******.saas.appdynamics.com -Dappdynamics.controller.port=443 -Dappdynamics.controller.ssl.enabled=true org.springframework.boot.loader.JarLauncher )

It can be seen that the MaxPermSize is set to 65M. Now, this is not something I configured for this app. This value was assigned by the Java build pack generated by Cloud Foundry when the app was deployed. This can be in the Java build pack’s OpenJDK JRE configuration.

# config/open_jdk_jre.yml
---
jre:
  version: 1.8.0_+
  repository_root: "{default.repository.root}/openjdk/{platform}/{architecture}"
memory_calculator:
  version: 1.+
  repository_root: "{default.repository.root}/memory-calculator/{platform}/{architecture}"
  memory_sizes:
    metaspace: 64m..
    permgen: 64m..
  memory_heuristics:
    heap: 75
    metaspace: 10
    permgen: 10
    stack: 5
    native: 10

The only way to change this value is to change the configurations of the Java build pack. There are two ways to do this:

  1. Fork the build pack, modify the default memory parameters and use this build pack to deploy.
  2. Override the memory parameters used by the build pack at deploy time.

Option 1 seems to be an overkill and Cloud Foundry provides a relatively easy method to override build pack parameters. Option 2 is the approach I decided to take.

There are two sets of mappings that control the various Java memory configurations - memory_sizes and memory_heuristics. Together, these two parameters provide a lot of flexibility to control memory allocation. For the particular issue my application faced, all I need to do is to force an appropriate value for MaxPermSize and this can be done by setting the JBP_CONFIG_OPEN_JDK_MEMORY_CALCULATOR environment variable as:

cf set-env app-name JBP_CONFIG_OPEN_JDK_MEMORY_CALCULATOR [memory_sizes: {metaspace: 128m}]'

While this configurations works for applications running under Java 1.8, the right parameter to use for applications running under Java 1.7 is permgen.

cf set-env app-name JBP_CONFIG_OPEN_JDK_JRE '[memory_sizes: {metaspace: 128m}]'

After re-staging the application and making sure the application works as expected, I added this environment variable to the application’s manifest.yml file.

---
applications:
- name: test-app
  ...
  env:
    JBP_CONFIG_OPEN_JDK_MEMORY_CALCULATOR: "[memory_sizes: {metaspace: 128m}]"
  ...

To see how to adjust the memory parameters by modifying the Java build pack, see Haydon Ryan’s blog post here.

If you have questions or comments about this blog post, you can get in touch with me on Twitter @sdqali.

If you liked this post, you'll also like...