Skip to main content

21 — The test harness: containers, tenants, and honest exclusions

Read this first: this lesson is about running the suite, not about writing tests. By the end you should know which command to type for the thing you just changed, which tests will never run unless you type their names, and why the repository says so out loud. Every number below was measured today; the counts in docs/testing/ and in tests.yml are older and lower, and where they disagree with this page, re-measure rather than believe either of us.

Time: about 35 minutes. Assumes lesson 20.

The inventory, measured now​

Count it yourself before trusting a table — from the repository root:

find backend/src/test/java -name '*.java' | wc -l                 # 65 files
grep -rlE '@Test\b' backend/src/test --include='*.java' | wc -l   # 58 declare a test
grep -rhoE '@Test\b' backend/src/test --include='*.java' | wc -l  # 470 methods

The seven files that declare nothing are the abstract base class TenantTestSupport, five three-line stubs under controller/auth/, and one zero-byte file — the "Dead files in the tree" of ../testing/backend-tests.md. Where the 470 methods live:

AreaFiles with tests@Test methods
service/impl/26244
controller/ (including controller/auth/)858
tenancy/752
security/640
service/provisioning/323
service/billing/provider/218
mail/215
config/, util/, model/, mapper/420

The shape is the point: over half the suite sits on the payroll engine and the money math, because that is the code whose bugs become amended filings. PayrollServiceImplTest alone carries 70. Two footnotes on 470: it counts declared methods, and one @ParameterizedTest exists (in CertificateGeneratorTest), so Surefire reports slightly more — and it is not what mvn test runs.

The command you actually run​

Run it from backend/. mvn test at the repository root runs zero tests and still prints BUILD SUCCESS, because the root pom.xml is the dead JavaFX project with no <modules>. Note that -Dtest matches the class name, not the filename — which matters for exactly one file, where filename, directory and declared class all disagree.

cd backend
mvn test                                  # 436 methods: everything except the *IT classes
mvn test -Dtest='*IT'                     # the other 34, which nothing runs for you
mvn test -Dtest='!Tenant*Test'            # everything that needs no Docker daemon
mvn test -Dtest='PayrollServiceImplTest#create*'   # one class, methods by pattern
mvn clean test                            # after touching an entity or a migration

The *IT truth: nothing is bound to run them​

Search backend/pom.xml for surefire. Nothing. Search for failsafe. Also nothing. The <build> section declares exactly two plugins — maven-compiler-plugin, configured for the annotation processors from lesson 09, and spring-boot-maven-plugin — so how tests get selected is stock Surefire behaviour, not a decision this repository made.

Stock Surefire includes **/Test*.java, **/*Test.java, **/*Tests.java and **/*TestCase.java. Read those four against RowLevelSecurityIT.java and the consequence follows mechanically: it matches none. The plugin that does claim *IT — Failsafe — appears only in the Spring Boot parent's pluginManagement, which declares a version and binds nothing.

Four files are affected, all in tenancy/: TenantIsolationIT (6 tests), RowLevelSecurityIT (5), TenantProvisioningIT (14) and RecognitionTenantIsolationIT (9). Thirty-four tests. So:

mvn test does not run them. mvn verify does not run them. CI does not run them.

They pass — run the command above and watch. But nothing runs it for you, so a change that breaks tenant isolation merges green. The reference page says so in the same flat words rather than burying it, which is the only reason you can plan around it.

Predict: rename RowLevelSecurityIT to RowLevelSecurityTest. What changes in CI, and is it an improvement?

It starts running on every pull request, with no other edit, because it now matches **/*Test.java, and the isolation gap closes for that file. What you buy it with is Docker becoming load-bearing for more of the gate — CI already needs a daemon for two classes (below), so this is less a new dependency than a bigger blast radius behind one point of failure — and speed, since the *IT classes are the slow self-managing ones. The current arrangement is a trade written down instead of assumed, which is the difference between a gap and a bug.

Why the tenancy tests need a real database​

The comment above the Testcontainers dependencies in backend/pom.xml is the whole argument:

        <!--
            Tenant isolation cannot be proven against mocked repositories: the thing under test is
            the SQL Hibernate emits. These give the tenancy tests a real Postgres with the real
            Flyway migrations applied. See TenantTestSupport.
        -->

A repository that has forgotten its tenant predicate returns exactly what the mock was told to return, and no assertion you write against a mock notices — the one place in this backend where the mocked style genuinely cannot see the bug. Everything else stays mocked; verify it, because grep -rn '@SpringBootTest' backend/src/test returns nothing and so does @WebMvcTest, and 35 files carry @ExtendWith(MockitoExtension.class). The only Spring context in the suite is one slice:

@DataJpaTest
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
// The @DataJpaTest slice pulls in Hibernate but not Flyway, and ddl-auto is validate, so without
// this the context fails against an empty schema instead of the migrated one we want to test.
@ImportAutoConfiguration(FlywayAutoConfiguration.class)
// The @DataJpaTest slice scans entities and repositories, not @Components, so without this the
// resolver bean is absent and Hibernate refuses to open a session at all ("configured for
// multi-tenancy, but no tenant identifier specified").
@Import(TenantIdentifierResolver.class)
@ActiveProfiles("tenancy-test")
public abstract class TenantTestSupport {

Read those four as a stack. replace = NONE says do not swap in an in-memory database; the @ImportAutoConfiguration puts the real migrations back, because the slice dropped them; the @Import puts back the lesson 12 bean the slice's component scan skipped. And the profile pins ddl-auto: validate in application-tenancy-test.yml, for the reason its comment gives: "the schema comes from Flyway and Hibernate only checks the mapping agrees with it. A tenant column added to a migration but not to its entity fails here."

The container starts itself, and that is deliberate​

The container is one field — @ServiceConnection static final PostgreSQLContainer POSTGRES = new PostgreSQLContainer("postgres:16-alpine") — with no @Testcontainers and no @Container anywhere. The Javadoc explains why:

Started once for the JVM rather than managed by @Testcontainers, which stops the container when its declaring class finishes -- and since this field is inherited, that would leave every subclass after the first talking to a dead container while Spring happily reused the cached context pointing at it. Ryuk reaps the container when the JVM exits.

That is a long afternoon: the first tenancy class passes, every one after it fails against a database that is gone, and Spring's context cache insists nothing changed. A static initializer sidesteps the lifecycle entirely. The version pin is not incidental either — the migrations use version-gated syntax (non-rewriting ADD COLUMN ... DEFAULT, later NULLS NOT DISTINCT), so a different major would be a different schema.

Two classes in the default run extend this base, TenantCoverageTest and TenantHarnessTest, so without a working Docker daemon plain mvn test fails — as ExceptionInInitializerError in TenantTestSupport, everything after it reporting NoClassDefFoundError, none of it mentioning Docker. If you see that cascade, run docker info before hunting for a code bug.

Two rules in the base class that look like fussiness​

Predict: the base class sets TenantContext.GLOBAL from a method annotated @BeforeTransaction, not @BeforeEach. Why can't it be @BeforeEach?

Has to be @BeforeTransaction rather than @BeforeEach: Spring starts the transaction -- and with it the Hibernate session that asks for a tenant -- before any @BeforeEach runs, so setting the context there is already too late.

Ordering is the answer. Spring's test transaction opens first; opening it opens a Hibernate session; a multi-tenant session asks the resolver for a tenant at that moment. By the time @BeforeEach fires you are inside a session that already failed to find one. The second rule is flushAndClear, whose comment is the reason isolation tests look the way they do:

Isolation assertions are worthless against a warm first-level cache: find() would hand back the instance this test just wrote without ever issuing the query whose tenant predicate is the thing under test.

If a tenancy assertion passes suspiciously fast, ask whether a query was issued at all. rawCount is the counterpart — a JDBC row count bypassing Hibernate, the ground truth a filtered query is compared against.

The best test in the repository​

TenantCoverageTest walks the JPA metamodel: for every entity Hibernate knows about it resolves the table name and checks it appears in exactly one of two curated sets, GLOBAL_TABLES or TENANT_OWNED_TABLES. An entity in neither fails the build:

        assertThat(unclassified)
                .as("New entities must be added to GLOBAL_TABLES or TENANT_OWNED_TABLES in this "
                        + "test. Ask: if two companies used this system, would they share these "
                        + "rows? Almost always the answer is no, and the table needs a tenant_id.")
                .isEmpty();

That message is a design document that breaks the build. It does not tell you what you got wrong; it asks the question you skipped. The class's Javadoc names the failure mode it exists to prevent: "someone adds an entity in a year's time, never thinks about tenancy, and it becomes a table every tenant shares. Nothing else in the build would notice -- the feature works perfectly in testing, because testing happens with one tenant."

Five more tests hold the classification honest from other directions: the lists match the columns the migrations created, the statutory tables stayed global, every tenant-owned entity is filtered by Hibernate rather than by hand, Role keeps a nullable discriminator so blueprints can exist, and the lists are disjoint. Each exception carries its reason inline. This is lesson 12's TenantOwned marker paying off: because the marker exists, a test can ask which entities carry it.

The two *IT classes worth reading​

RowLevelSecurityIT proves what the application layer cannot. Its Javadoc names the trap:

It connects as app_runtime rather than through the pool, because the owner role the tests otherwise use is a superuser and superusers bypass row-level security entirely -- a test running as the owner would pass no matter what the policies said.

That is the difference between testing RLS and testing nothing. Its assertions then leave the predicate out on purpose — // Exactly the mistake this guards against: no WHERE tenant_id anywhere. — and the rows still do not come back, while a companion test confirms the owner does still see everything so Flyway can keep migrating. That is lesson 13, executable.

TenantProvisioningIT covers the other shape. It runs @Transactional(propagation = Propagation.NOT_SUPPORTED) and manages its own transactions, because "provisioning spans three of them, in two different tenant scopes, which a single test-managed transaction could not reproduce." A @TestConfiguration supplies the beans the slice will not, among them a MailService that is a one-line lambda appending to a list — it records the welcome email rather than re-testing rendering.

House style, in one screen​

None of this is enforced by a plugin — only by consistency, so match it.

  • Naming. <ClassUnderTest>Test in the package mirroring the subject; methods read methodUnderTest_condition_expectedOutcome, so a failure names the broken rule without anyone opening the file. 263 method names carry that shape; 172 tests add a @DisplayName on top.
  • Construction. @ExtendWith(MockitoExtension.class), collaborators as @Mock, subject as @InjectMocks. Flat @Test methods — @Nested appears zero times.
  • AssertJ only. 927 assertThat( call sites, and not one import of org.junit.jupiter.api.Assertions anywhere in the tree. Money with isEqualByComparingTo, never isEqualTo — 257 call sites depend on it, because BigDecimal.equals compares scale and would make 1000 and 1000.0000 unequal.
  • Time is injected, never read. LocalDate.now() appears zero times. SignupVerificationServiceTest states the rule: "Time is moved, never slept through: the service takes a Clock, and each test swaps it for one offset into the future."
  • Real mappers, not mocked ones. Twelve files declare the generated MapStruct implementation as a @Spy, each carrying the same one-line comment: // The real generated mapper, not a mock: the mapping itself stays under test. — lesson 09's rule, in the suite.
  • Mockito is strict. A when(...) that never fires fails the test even when every assertion passes, which usually means the path short-circuits before the collaborator. Delete the stub rather than reaching for lenient strictness — two files use it, both documented compromises.

What CI does about the container​

tests.yml runs mvn -f backend/pom.xml test -B on every pull request to main/master, and deploy.yml calls the same workflow as its first job. Before the test step it pulls the Postgres image, three attempts with a growing backoff. Its comment says why a plain step would not do:

Nothing in that output mentions Docker or a registry, and the same commit passes locally, so it reads as a code failure and is not one.

A cold pull or an anonymous rate-limit happens inside Testcontainers' readiness window, surfaces in a static initializer, and cascades through every class that inherits the field. Pre-pulling moves the work off that clock; if the tag changes, change it in both places. Two comments there are already stale — "373 tests" and "Three classes drive real Postgres containers", where the default run is now 436 methods across two. A count in a comment is a snapshot, not a fact.

Where it is thin​

No coverage tool is configured — no JaCoCo, and no Surefire block at all; for a number, run the agent ad hoc using the command in ../testing/backend-tests.md. That page also keeps the ranked list of untested services, plus the two blind spots the mocked style buys its speed with: repository @Query behaviour and @PreAuthorize enforcement are never exercised. Read it there — it moves, and it is maintained. Know both before you call the backend tested.

Where this shows up in MotorPH​

Recap​

  • cd backend first, and re-measure before quoting. Today: 470 @Test methods in 58 files, of which mvn test executes 436.
  • *IT runs only when you name it. No Surefire and no Failsafe configuration exists in backend/pom.xml, so 34 tenancy tests fall outside stock include patterns — type mvn test -Dtest='*IT' before merging anything touching tenancy, entities or migrations.
  • Mocks cannot see SQL, which is why two classes start a real postgres:16-alpine from a static initializer while the rest of the suite starts no Spring context at all.
  • A test can be a design document. TenantCoverageTest fails the build with a question rather than a diff; RowLevelSecurityIT connects as the runtime role because a superuser would pass.
  • Match the house style without being told: @InjectMocks under strict Mockito, AssertJ only, money via isEqualByComparingTo, time via an injected Clock, real mapper as a @Spy.

Next: 22 — The map and the gaps.