tree 20c3fa8c4833324b3306ec4e50345aa95a19c545
parent c57dfe5951ef9afcb6c4516ffcb34c67a7eb3e15
author John Johansen <john.johansen@canonical.com> 1666685921 -0700
committer Kevin Berry <kpberry@google.com> 1755830481 -0700

apparmor: cache buffers on percpu list if there is lock contention

[ Upstream commit ea9bae12d02819556db63348db8bd8441eb316f2 ]

commit df323337e507 ("apparmor: Use a memory pool instead per-CPU caches")

changed buffer allocation to use a memory pool, however on a heavily
loaded machine there can be lock contention on the global buffers
lock. Add a percpu list to cache buffers on when lock contention is
encountered.

When allocating buffers attempt to use cached buffers first,
before taking the global buffers lock. When freeing buffers
try to put them back to the global list but if contention is
encountered, put the buffer on the percpu list.

The length of time a buffer is held on the percpu list is dynamically
adjusted based on lock contention.  The amount of hold time is
increased and decreased linearly.

v5:
- simplify base patch by removing: improvements can be added later
  - MAX_LOCAL and must lock
  - contention scaling.
v4:
- fix percpu ->count buffer count which had been spliced across a
  debug patch.
- introduce define for MAX_LOCAL_COUNT
- rework count check and locking around it.
- update commit message to reference commit that introduced the
  memory.
v3:
- limit number of buffers that can be pushed onto the percpu
  list. This avoids a problem on some kernels where one percpu
  list can inherit buffers from another cpu after a reschedule,
  causing more kernel memory to used than is necessary. Under
  normal conditions this should eventually return to normal
  but under pathelogical conditions the extra memory consumption
  may have been unbouanded
v2:
- dynamically adjust buffer hold time on percpu list based on
  lock contention.
v1:
- cache buffers on percpu list on lock contention

BUG=b/437414032,b/440373491
TEST=presubmit, go/sql-apparmor-spinlock-fix-verification
RELEASE_NOTE=None

cos-patch: bug
Reported-by: Sergey Senozhatsky <senozhatsky@chromium.org>
Reviewed-by: Georgia Garcia <georgia.garcia@canonical.com>
Change-Id: I90dba49409979dfac00c820ba063d38256084d2a
Signed-off-by: John Johansen <john.johansen@canonical.com>
Reviewed-on: https://cos-review.googlesource.com/c/third_party/kernel/+/109265
Tested-by: Cusky Presubmit Bot <presubmit@cos-infra-prod.iam.gserviceaccount.com>
Main-Branch-Verified: Cusky Presubmit Bot <presubmit@cos-infra-prod.iam.gserviceaccount.com>
Reviewed-by: Robert Kolchmeyer <rkolchmeyer@google.com>