With the continued growth and transition to microservices, it’s important to ensure that the time and money re-engineering systems to modern, cloud-based solutions lead to tangible benefits to the organization. In this multi-part series, we’ll look at different components and pitfalls that need to be considered when modernizing to microservices. 

In this blog, we’ll look at Memory Management between Java and Kubernetes.

Understanding Memory Configuration Challenges Between Kubernetes And Java

There seems to be a lot of confusion about Kubernetes memory requests and limit in application development teams. I think it’s because people tend to incorrectly map the new concepts to the Java memory settings: the initial heap size is being confused with the pod-requested memory, and the maximum heap size is thought to map to the pod memory limit. This is incorrect. Kubernetes will only guarantee that you have access to the requested amount of memory. The guarantee on the limit is that if you request more, the pod will be killed – it’s not a guarantee of availability up to that limit. Actually, the pod scheduler does not even look at the memory limit to schedule a pod on a particular node. A compounding factor in my opinion is that recent Java versions (8u191+) allocate by default 25% of the container’s memory limit to the heap. This increases the likelihood that the pod gets killed as the limit is not guaranteed, and that’s probably why the percentage is so low (but nonetheless using the memory limit of the container by default, otherwise increasing the limit and leaving the request untouched would have had no impact on the Java heap size).

Reduce Moving Parts through Simplification

To reduce the moving parts and the variability of the different dimensions, I suggest simplifying the configuration of memory settings to the following:

1. Set the Java heap initial and max size to be the same value. Use a specific number for memory and not a percentage of the available memory of the container (MiB or GiB), so that it is cognitively simpler than having a percentage for Java’s heap vs. a number for the pod memory.
2. Set the pod request and limit to the same value. This value needs to be higher than the Java heap size, but the same number for both the request and limit. The extra memory at the pod level maps to the non-heap components of Java (stack frames, classloader, etc.). It also includes Java’s Direct Memory which is used for memory accessible not just by the JVM but also by the system, providing optimized IO, in certain cases. This was found to be used by Redis/Netty in our case, and complicated the understanding of the memory usage as being outside the JVM’s heap (it is controllable by using -XX:MaxDirectMemorySize).

In the longer term, there needs to be proper visibility into the cost to run the microservices. If the product team is unaware of how much their service costs to run (in memory requirements, yes, but that also applies to compute and storage), the incentive loop to optimize operational cost can be nonexistent. These metrics need to be collected and available to the products team, helping to justify the value of spending time on that technical debt.

Need to catch up? Previously, lessons included:

Part 1: The Importance of Starting with the Team

Part 2: Defining Ownership

Part 3: Process Management and Production Capacity

Part 4: Reserving Capacity for Innovation

Part 5: Microservices Communication Patterns

Part 6: Using Shadow Release Strategy

Part 7: Performance Testing Microservices

In Part 9, we’ll be looking at how to prioritize testing within microservices.

Ready to modernize your organization’s microservices? Oteemo is a leader in cloud-native application development. Learn more: https://oteemo.com/cloud-native-application-development/