This week I have been trying out Cloud Foundry. Today while trying to set up monitoring through AppDynamics, the sample application that I was using started crashing because of a Java memory error. This blog post discusses how to prevent this from happening by configuring Java memory parameters used by the application.
This is what was in the Cloud Foundry logs.
May 20 18:14:51 CloudFoundry 6767b9fd-1874-43cb-a4f8-7470a17c90ae/[App/2]: /bin/bash: line 31: 32 Killed ( SERVER_PORT=$PORT $PWD/.java-buildpack/open_jdk_jre/bin/java -cp $PWD/.:$PWD/.java-buildpack/spring_auto_reconfiguration/spring_auto_reconfiguration-1.4.0_RELEASE.jar -Djava.io.tmpdir=$TMPDIR -XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh -Xmx499200K -Xms499200K -XX:MaxPermSize=65M -XX:PermSize=65M -Xss1M -javaagent:$PWD/.java-buildpack/app_dynamics_agent/javaagent.jar -Dappdynamics.agent.applicationName='******' -Dappdynamics.agent.tierName='******' -Dappdynamics.agent.nodeName=$(expr "$VCAP_APPLICATION" : '.*instance_id[": ]*"\([a-z0-9]\+\)".*') -Dappdynamics.agent.accountAccessKey=****** -Dappdynamics.agent.accountName=******* -Dappdynamics.controller.hostName=******.saas.appdynamics.com -Dappdynamics.controller.port=443 -Dappdynamics.controller.ssl.enabled=true org.springframework.boot.loader.JarLauncher )
It can be seen that the MaxPermSize
is set to 65M
. Now, this is not something I configured for this app. This value was assigned by the Java build pack generated by Cloud Foundry when the app was deployed. This can be in the Java build pack’s OpenJDK JRE configuration.
# config/open_jdk_jre.yml
---
jre:
version: 1.8.0_+
repository_root: "{default.repository.root}/openjdk/{platform}/{architecture}"
memory_calculator:
version: 1.+
repository_root: "{default.repository.root}/memory-calculator/{platform}/{architecture}"
memory_sizes:
metaspace: 64m..
permgen: 64m..
memory_heuristics:
heap: 75
metaspace: 10
permgen: 10
stack: 5
native: 10
The only way to change this value is to change the configurations of the Java build pack. There are two ways to do this:
- Fork the build pack, modify the default memory parameters and use this build pack to deploy.
- Override the memory parameters used by the build pack at deploy time.
Option 1 seems to be an overkill and Cloud Foundry provides a relatively easy method to override build pack parameters. Option 2 is the approach I decided to take.
There are two sets of mappings that control the various Java memory configurations - memory_sizes
and memory_heuristics
. Together, these two parameters provide a lot of flexibility to control memory allocation. For the particular issue my application faced, all I need to do is to force an appropriate value for MaxPermSize
and this can be done by setting the JBP_CONFIG_OPEN_JDK_MEMORY_CALCULATOR
environment variable as:
cf set-env app-name JBP_CONFIG_OPEN_JDK_MEMORY_CALCULATOR [memory_sizes: {metaspace: 128m}]'
While this configurations works for applications running under Java 1.8
, the right parameter to use for applications running under Java 1.7
is permgen
.
cf set-env app-name JBP_CONFIG_OPEN_JDK_JRE '[memory_sizes: {metaspace: 128m}]'
After re-staging the application and making sure the application works as expected, I added this environment variable to the application’s manifest.yml
file.
---
applications:
- name: test-app
...
env:
JBP_CONFIG_OPEN_JDK_MEMORY_CALCULATOR: "[memory_sizes: {metaspace: 128m}]"
...
To see how to adjust the memory parameters by modifying the Java build pack, see Haydon Ryan’s blog post here.
If you have questions or comments about this blog post, you can get in touch with me on Twitter @sdqali.
If you liked this post, you'll also like...
- Implementing feature toggles for a Spring Boot application - Part 4
- Implementing feature toggles for a Spring Boot application - Part 3
- Implementing feature toggles for a Spring Boot application - Part 2
- Implementing feature toggles for a Spring Boot application - Part 1
- Setting up a secure etcd cluster behind a proxy
- Handling Deserialization errors in Spring Redis Sessions
- CSRF Protection with Spring Security and Angular JS
- Controlling Redis auto-configuration for Spring Boot Session
- JWT authentication with Spring Web - Part 5
- JWT authentication with Spring Web - Part 4
- JWT authentication with Spring Web - Part 3
- JWT authentication with Spring Web - Part 2
- JWT authentication with Spring Web - Part 1
- JSON logging for Spring applications
- Injecting dependencies into a Spring @Configuration
- Filtering responses in Spring MVC
- Deprecating domain events in Axon
- Programmable exit codes for spring command line applications - 2
- Programmable exit codes for Spring command line applications
- Using custom arguments in Spring MVC controllers
- Authentication for Apache Camel HTTP components
- Thoughts on Open Graph tags
- Integration testing Spring command line applications
- Integration testing challenges for non-web Spring applications
- How thinking of Documentation as Legislation helped me become a better programmer
- Implementing custom annotations for Spring MVC
- Validating RequestParams and PathVariables in Spring MVC
- Testing async responses using MockMvc
- Running multiple applications in the same Tomcat installation
- Making sense of Cloud Foundry security group declarations
- Clojure Dojo - Levenshtein edit distance
- A simple JMeter test with login
- Implementing Rate Limiting in Rails - Part 2
- Implementing Rate Limiting in Rails - Part 1
- Python Hack - Dynamically override an object's attribute
- Managing Gemsets in Rbenv
- Looking up Compiler params used to compile a Ruby version
- Python's bool type
- Validating JSON in Emacs
- Emacs hack: Viewing Git logs while composing commit messages
- Configure Git's comment character
- My experience working remotely
- Oh I can build it in...
- JavaScript, clipboard access and hidden flash widgets
- Reducing Emacs startup time while committing
- My first Firefox plugin: GetCache - View cached version of the current page
- GetCache - A Chrome plugin to view cached version of the current page
- On REST, Content-Type, Google Chrome and Caching
- Understanding Python's "with" statement
- Heredocs in Ruby and Python
- Micro Journal - simple Git-backed journal in Python
- VodQA NCR: Maintaining Large Test Suites
- Know Your Tools - Don't Shoot Yourself in the Foot
- Managing security certificates from the console - on Windows, Mac OS X and Linux
- Indian and Pakistani cricketers - who make better debuts?
- Fixing Flyspell for Emacs in Mac OS X
- Finding un-merged commits with git cherry
- Bullet proof Jenkins setup
- Why your project should have a Getting Started guide.
- Debugging: C Sharp's HttpWebRequest, 100-Continue and nginx
- Wikipedia Page Hopping
- Empathy Log Parser
- Binary Signature Art
- Java Arrays in JRuby
- Autorun.py - Execute stuff on file change