A pod can start using excessive memory for many reasons. Perhaps frequent traffic causes bursts in memory that isn’t reclaimed fast enough. Or maybe running the software for long periods of time causes corruption that slowly builds. Maybe the app is running a big query or uploading a large file and caching all the data in memory along the way.
Whatever the cause, when memory climbs, the software becomes less responsive, and other containers scheduled on this node also suffer. In most cases, a pod restart can fix the symptom, but mask the problem. Faulty software or bad data will build up again, and in time the pod will restart again.
If a container isn’t functioning as expected, this leads to degraded user experience. If all the containers in a specific microservice fail, the site can go offline completely.
Shoreline makes it easy to detect excessive memory use in a container. Shoreline compares the pod’s resource limits to the memory currently in use. If the memory spikes beyond the limit, Shoreline captures diagnostic details around the application including memory and CPU used, and JVM details if applicable, terminates the pod, and lets Kubernetes automatically restart it. If this alert is flagged too often, the user can adjust the pod’s resource limits.
