Stateful testing¶
Stateful testing introduces state machines.
Class-based state machines¶
- class hegel.StateMachine¶
Base class for class-based state machines.
A state machine test checks a system through sequences of operations: each test case runs a sequence of rules chosen by the engine, checking the invariants along the way, and a failure is shrunk to a short sequence of steps. Rules and invariants are methods decorated with
rule()andinvariant(); each takes the test case for its step. The class attributessteps,min_concurrencyandmax_concurrencyconfigure the run, as theMachinemethods of the same names do; withmax_concurrency > 1the rules run on worker threads against the shared instance. A fresh instance is made for every test case:__init__takes either no arguments or just the test case, to draw the initial state from.class Counter(hegel.StateMachine): def __init__(self): self.count = 0 @hegel.rule() def incr(self, tc): n = tc.draw(gs.integers(min_value=1, max_value=10)) self.count += n @hegel.invariant() def non_negative(self, tc): assert self.count >= 0 test_counter = hegel.state_machine_test(Counter)
- hegel.rule(fn: Callable[[...], Any] | None = None, *, groups: Iterable[str] | None = None, weight: float = 1.0, name: str | None = None) Any¶
Mark a
StateMachinemethod as a rule: one step the engine may choose to run. The method takes the test case and draws whatever the step needs:@hegel.rule() def push(self, tc): self.stack.append(tc.draw(gs.integers()))
Usable as
@hegel.ruleor@hegel.rule(...).namereplaces the method name in the failure report;groupsandweightare as forMachine.rule(). A rule’s precondition is aTestCase.assumeat its top.
- hegel.invariant(fn: Callable[[...], Any] | None = None, *, always_run: bool = False, pause_faults: bool = False, name: str | None = None) Any¶
Mark a
StateMachinemethod as an invariant: a check of the machine’s state that fails by raising. The method takes the test case:def valid(self, tc).Every invariant runs on the initial and final state. Hegel samples invariants at intermediate steps. Set
always_run=Trueto check the invariant after every step. Withpause_faults=True, fault injection is paused for the duration of each check: inside Antithesis the fuzzer stops injecting faults while the invariant runs; elsewhere the flag has no effect. Usable as@hegel.invariantor@hegel.invariant(...);namereplaces the method name in the failure report.
- hegel.state_machine_test(machine: type[StateMachine], **settings: Any) Callable[[...], None]¶
A test function that runs
machine, aStateMachinesubclass, for pytest to collect when assigned to atest_name:test_counter = hegel.state_machine_test(Counter, test_cases=200)
Keyword arguments are settings, as for
test(). A failure report says how to replay the failure by wrapping this call inreproduce_failure().
- hegel.run_state_machine(tc: TestCase, machine: type[StateMachine] | StateMachine | Machine, **overrides: Any) None¶
Run a state machine on
tc: aMachine, aStateMachineinstance, or a class to instantiate (withtcif its constructor accepts one).steps,min_concurrencyandmax_concurrencymay be overridden; any other keyword is anInvalidArgument. Use it to run a machine from inside atest(), after other setup:@hegel.test def test_counter(tc): hegel.run_state_machine(tc, Counter, steps=100)
Function-based state machines¶
- class hegel.Machine(tc: TestCase)¶
A state machine assembled from rule and invariant callables, inside a test: the function-based counterpart of
StateMachine.@hegel.test def test_stack(tc): stack = [] machine = hegel.Machine(tc) machine.rule("push", lambda tc: stack.append(tc.draw(gs.integers()))) @machine.rule("pop") def pop(tc): tc.assume(stack) stack.pop() @machine.invariant("bounded") def bounded(tc): assert len(stack) <= 50 machine.run()
Each rule and invariant takes the test case for its step, and draws from it.
steps(),min_concurrency()andmax_concurrency()return the machine, so calls chain.With
max_concurrencyabove 1 the machine runs concurrently, in rounds, onmax_concurrencyworker threads. The engine draws how many of them run rules in[min_concurrency, max_concurrency], weighted toward the maximum, and may redraw it between rounds. Each round, it picks one group of rules, every active worker runs a short sequence of that group’s rules against the shared model, and the invariants then run on the main thread. Rules in the same group may overlap; rules in different groups never do; a rule in several groups runs in the rounds of each of them, so areadrule in both awriteand aflushgroup can overlap either. A concurrent run’s failures may not reproduce exactly, so they are confirmed by replay and reported with a caveat; seeNondeterminismStrictness.- rule(name: str, apply: Callable[[TestCase], Any] | None = None, *, groups: Iterable[str] | None = None, weight: float = 1.0) Any¶
Add a rule called
namethat runsapply(tc)as one step; usable as a decorator whenapplyis omitted.A rule that rejects (with
TestCase.assume) stops early without failing; rules should reject before they change the model.groupsnames the rule’s concurrency groups: rules in the same group may overlap, rules in different groups never do, and a rule in several groups runs in the rounds of each of them (rules without one share the anonymous group).weightis a positive hint of how often to choose the rule relative to the others. A duplicate name, a weight that is not finite and positive, or a single string in place of the list ofgroupsis anInvalidArgument; so are an emptygroupsand one that repeats a group, which the engine reports when the machine runs.
- invariant(name: str, apply: Callable[[TestCase], Any] | None = None, *, always_run: bool = False, pause_faults: bool = False) Any¶
Add an invariant called
name, checked byapply(tc); usable as a decorator whenapplyis omitted.Every invariant runs on the initial and final state. Hegel samples invariants at intermediate steps. Set
always_runto check the invariant after every step. Withpause_faultsset, fault injection is paused for the duration of each check: inside Antithesis the fuzzer stops injecting faults while the invariant runs, so it observes the system under test without a fault landing in the middle of the check; elsewhere the flag has no effect. A duplicate name is anInvalidArgument.
- steps(step_count: int) Machine¶
Set the number of steps per test case (default 50): every test case runs at least one and at most
step_count, most run them all, and shrinking shortens a failing one. A step is one completed rule (one that rejects does not count) or, for a concurrent machine, one round. Returns the machine.
- min_concurrency(n: int) Machine¶
Set the fewest worker threads a concurrent run uses (default 1). Returns the machine.
- max_concurrency(n: int) Machine¶
Set the most worker threads a run uses (default 1, sequential). Above 1 the machine runs concurrently. Returns the machine.
- run() None¶
Run the machine on this test case: invariants on the initial state, then steps chosen by the engine, then invariants on the final state. The failure report lists each step’s rule and draws. A machine with no rules, or with
min_concurrencyabovemax_concurrency, is anInvalidArgument.
Pools¶
- class hegel.Pool(tc: TestCase)¶
A pool of values a stateful test accumulates, with generators that draw from it:
values_reusable()leaves the value in the pool,values_consumed()removes it. Drawing from an empty pool rejects the test case (so the rule stops early).The engine tracks which value is drawn, so shrinking keeps the values a failure depends on and drops the rest; it shrinks toward values added earlier. Create the pool from the test case, typically in the machine’s
__init__:class Files(hegel.StateMachine): def __init__(self, tc): self.handles = hegel.Pool(tc) @hegel.rule() def open(self, tc): self.handles.add(open_file(tc.draw(gs.text(min_size=1)))) @hegel.rule() def close(self, tc): tc.draw(self.handles.values_consumed()).close()
len(pool)is the number of values in it.- add(value: Any, tc: TestCase | None = None) None¶
Add
valueto the pool.tcis the test case of the step adding it, by default the one the pool was created with.
- class hegel.ConcurrentPool(tc: TestCase)¶
A
Poolsafe to share between the worker threads of a concurrent state machine.add()must be given the worker’s test case (the one the rule received), or it is anInvalidArgument.