Casbin
Advanced usage

Benchmarks

Overhead of Policy Enforcement in Casbin

5 min read

Policy enforcement overhead is benchmarked in model_b_test.go. Testing environment:

Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz, 2601 Mhz, 4 Core(s), 8 Logical Processor(s)

Results from go test -bench=. -benchmem (op = one Enforce() call, ms = millisecond, KB = kilobytes):

Test caseRule sizeTime overhead (ms/op)Memory overhead (KB)
ACL2 rules (2 users)0.0154935.649
RBAC5 rules (2 users, 1 role)0.0217387.522
RBAC (small)1100 rules (1000 users, 100 roles)0.16430980.620
RBAC (medium)11000 rules (10000 users, 1000 roles)2.258262765.152
RBAC (large)110000 rules (100000 users, 10000 roles)23.9167767,606
RBAC with resource roles6 rules (2 users, 2 roles)0.0211467.906
RBAC with domains/tenants6 rules (2 users, 1 role, 2 domains)0.03269610.755
ABAC0 rule (0 user)0.0075102.328
RESTful5 rules (3 users)0.04539891.774
Deny-override6 rules (2 users, 1 role)0.0232818.370
Priority9 rules (2 users, 2 roles)0.0163895.313

Policy enforcement overhead in Pycasbin is benchmarked in tests/benchmarks. Testing environment:

Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz (Run by Github actions)
platform linux -- Python 3.11.4, pytest-7.0.1, pluggy-1.2.0

Results from executing casbin_benchmark (op = one enforce() call, ms = millisecond):

Test caseRule sizeTime overhead (ms/op)
ACL2 rules (2 users)0.067691
RBAC5 rules (2 users, 1 role)0.080045
RBAC (small)1100 rules (1000 users, 100 roles)0.853590
RBAC (medium)11000 rules (10000 users, 1000 roles)6.986668
RBAC (large)110000 rules (100000 users, 10000 roles)77.922851
RBAC with resource roles6 rules (2 users, 2 roles)0.106090
RBAC with domains/tenants6 rules (2 users, 1 role, 2 domains)0.103628
ABAC0 rule (0 user)0.053213
RESTful5 rules (3 users)NA
Deny-override6 rules (2 users, 1 role)NA
Priority9 rules (2 users, 2 roles)0.084684

Policy enforcement overhead in Casbin CPP is benchmarked in tests/benchmarks using Google's benchmarking tool. Testing environment:

Intel(R) Core(TM) i5-6300HQ CPU @ 2.30GHz, 4 cores, 4 threads

Results from executing the casbin_benchmark target built in Release configuration (op = one enforce() call, ms = millisecond):

Test caseRule sizeTime overhead (ms/op)
ACL2 rules (2 users)0.0195
RBAC5 rules (2 users, 1 role)0.0288
RBAC (small)1100 rules (1000 users, 100 roles)0.300
RBAC (medium)11000 rules (10000 users, 1000 roles)2.113
RBAC (large)110000 rules (100000 users, 10000 roles)21.450
RBAC with resource roles6 rules (2 users, 2 roles)0.03
RBAC with domains/tenants6 rules (2 users, 1 role, 2 domains)0.041
ABAC0 rule (0 user)NA
RESTful5 rules (3 users)NA
Deny-override6 rules (2 users, 1 role)0.0246
Priority9 rules (2 users, 2 roles)0.035

Policy enforcement overhead in Lua Casbin is benchmarked in bench.lua. Testing environment:

AMD Ryzen(TM) 5 4600H CPU @ 3.0GHz, 6 Cores, 12 Threads

Results from running luajit bench.lua (op = one enforce() call, ms = millisecond):

Test caseRule sizeTime overhead (ms/op)
ACL2 rules (2 users)0.0533
RBAC5 rules (2 users, 1 role)0.0972
RBAC (small)1100 rules (1000 users, 100 roles)0.8598
RBAC (medium)11000 rules (10000 users, 1000 roles)8.6848
RBAC (large)110000 rules (100000 users, 10000 roles)90.3217
RBAC with resource roles6 rules (2 users, 2 roles)0.1124
RBAC with domains/tenants6 rules (2 users, 1 role, 2 domains)0.1978
ABAC0 rule (0 user)0.0305
RESTful5 rules (3 users)0.1085
Deny-override6 rules (2 users, 1 role)0.1934
Priority9 rules (2 users, 2 roles)0.1437

Benchmark monitoring

The embedded page below displays Casbin's performance across commits.

How is this guide?

On this page