Stopping jobs from consuming too much CPU

Foundation monitors the activity level of jobs. The activity level is the number of status changes per second and a status change is a transition from one activity to another, e.g. reading different database records, communicating with the client and calling sub-programs. Jobs that have had a high activity level for a certain amount of time will get their CPU priority lowered, to ensure that these jobs do not starve other jobs from CPU usage.

This mechanism is governed by two properties:

  • m3.job.activityLevelThreshold: Watch jobs with a sustained activity level over this level and adjust job priority after [m3.job.highActivityLevelTimeLimitMinutes] minutes have passed. The default value is 999.

  • m3.job.highActivityLevelTimeLimitMinutes: The number of minutes a job can have an activity level over [m3.job.activityLevelThreshold] before the job priority is adjusted. The default is 1 minute.

An MvxJob is in one of three states during its lifetime; Running, Observed and Lowered.

  • Running: The normal state. When the mean activity level crosses the threshold the new state is Observed.

  • Observed: If the high activity is sustained for more than the configured time, the priority is lowered to Thread.MIN_PRIORITY and the new state is lowered. If the activity level creeps below the threshold, the new state is Running again

  • Lowered: The state for LOW_PRIORITY threads. If the activity level creeps below the threshold, the new state is Running again.

All transitions and decisions are logged on trace on the MvxJob logger in the Subsystem module.