Chapter 11
Design of the Empirical Work: Connecting to Observation

11.1 What is to be tested

Part I defined quantities and propositions, and Part II deduced the subspaces that remain under constraints. Neither used observation. From this part onwards they are connected to observation.

What is connected divides in two.

The first is measurement, the second verification. Because they need different preparation, they are treated in stages below.

11.2 Five stages to pass

Connecting one quantity, or one proposition, to observation passes through five stages. The organization is this text’s own.

The first three are prerequisites in series: failing any one blocks the stages that follow. The last two are ways of using the values obtained, not a continuation of the sequence. They are separated by whether pre-registration has taken place.

The benefit of separating the stages is that the work required differs by where the process stalls. A gap at stage 1 is not filled by collecting any amount of data, and a gap at stage 3 is not filled by refining any number of definitions. Without first judging where the process is stuck, effort is misallocated.

11.3 What to check at each stage

11.3.1 Stage 1: operationalization

Three checks.

(1)

Does the quantity have a unit?

(2)

Is the object of observation fixed — the firm, a $\mathrm{\Phi }$, or an identifier?

(3)

For a proposition, is the sign uniquely fixed?

Some quantities of Part I are operationalized by their definitions and some are not. For the former, Chapter 11.5 lists the variables to be recorded for each $\mathrm{\Phi }$.

For the latter — quantities that are defined but whose unit or object of observation is not fixed — that has to be settled before proceeding to measurement. A defect at this stage cannot be detected by adding data. Values do come out, so a result appears without it being clear what was measured.

Item (3) is specific to propositions. If the theory does not fix the sign, either result is consistent with it and no test is possible.

11.3.2 Stage 2: population

Three checks.

(1)

Have the reference date, the scope and the lower bound been fixed?

(2)

What falls outside the sample, and is it correlated with the axis of interest?

(3)

Do objects that have stopped or exited remain in the sample?

Items (2) and (3) are the central theme of this part and are treated in Chapter 12. In this framework (3) weighs particularly heavily: the insolvency time of Section 4.2 depends on $\mathrm{\Phi }$, so an attempt to estimate $\mathrm{\Phi }$ makes the composition of the sample itself a function of the object of estimation.

11.3.3 Stage 3: measurement

Three checks.

(1)

Is the quantity measured anywhere?

(2)

If it is, can it be obtained at the granularity required?

(3)

If it cannot, which category of obstacle applies?

The judgment at (3) governs the design of the survey, because the category determines whether spending effort produces progress or nothing at all. Since the categories were constructed through the survey work itself, they are placed in Chapter 13.

11.3.4 Stages 4 and 5: how the values obtained are used

Passing stage 3 yields values. Their use divides in two, treated in the next section.

11.4 Reconciliation versus verification

Comparing a deduction with existing aggregates after the fact is called reconciliation. Judging an implication fixed in advance against data is called verification.

Reconciliation cannot be falsified. If the figures disagree one can say another factor was at work, and nothing in the sample rules that explanation out. Reconciliation therefore neither supports nor refutes the theory.

It still has a use. It confirms that a deduction is not wildly at odds with reality, and it helps in choosing what to verify next. It has value as a check and none as evidence.

The danger lies elsewhere.

The comparison in Chapter 8 between the revival of family 3 and the fields of business started by age is of this form. That section states explicitly that it is an after-the-fact reconciliation and shows no causation.

Hence the following discipline.

When a reconciliation is performed, say that it is one. Do not use it in place of an implication fixed in advance.

This has the same form as the rule in Chapter 12 that well-known firms are used for illustration and not for inference. Both require that material not be used beyond the strength of the evidence it carries.

11.5 Where each stage is handled in this text

Chapter 14 opens Part IV in order to tabulate the results of stage 3. Making explicit which quantities could not be measured is that chapter’s main purpose, and that is a record of where between stages 1 and 3 the process stalled.

11.6 Priorities for the survey

The objects are narrowed to families 2 and 5. The selection can be derived deductively.

${\varphi }_{\mathrm{prod}}$ does not depend on $\mathrm{\Phi }$. Of the three terms of (6.1), ${\varphi }_{\mathrm{prod}}=(p^{\ast }-c(\delta ))\delta$ is fixed by marginal cost and the competitive price and contains nothing about the shape of the map from delivery to settlement. Replacing $\mathrm{\Phi }$ alone, holding $C_{\mathrm{prod}}$ and marginal cost fixed, changes the sign of $\kappa$ and the working capital required, not the production surplus (Table 2.5).

The shape of $\mathrm{\Phi }$ bears on surplus through the other two terms. To test the framework, it suffices to choose $\mathrm{\Phi }$ that isolate each of those two terms.

The two terms arise from different degrees of freedom. ${\varphi }_{\mathrm{cog}}=p\cdot 𝔼[\overline{\delta }-\delta ]$ can be positive only when the contract states a ceiling $\overline{\delta }$, that is, when $\pi$ does not refer to $\delta$ (Remark 2.22). That is degree of freedom (5). Relating the divergence to the transaction frequency $\nu$ additionally requires the repetition of degree of freedom (6): in a one-off, $\overline{\delta }-\delta$ is a single realization with no distribution, so no relation to $\nu$ can be asked. The family in which both (5) and (6) are present is family 2, which Part II designates the principal home of cognitive surplus.

${\varphi }_{\mathrm{barg}}=(p-p^{\ast })\delta$ is a zero-sum transfer whose allocation is fixed by whether the $y$ in $\pi =h(y)$ is verifiable. That is degree of freedom (4). In the order of Table 7.8, the only family decided by degree of freedom (4) is family 5.

The two families sit diagonally in the quadrant diagram. Degrees of freedom (1) and (2) dominate, and those two axes give the four-way split of Figure 2.1. Family 2 is advance and flat ($\pi$ independent of $\delta$, divergence possible), family 5 is arrears and linked ($\pi$ tied to $\delta$, no divergence): both axes take opposite values. Choosing only one family would allow only one of the two axes to be varied.

What can be tested. Family 2 permits direct tests of cognitive surplus ((2.15)) and of the capacity proposition. In particular, that one and the same flat membership divides in two according to whether a capacity constraint is present is the clearest instance of outwardly similar $\mathrm{\Phi }$ having different durability.

Family 5 permits a direct test of the measurability condition (Chapter 4). The claim that success fees and cost plus transfer risk in opposite directions is testable as a correspondence between contract form and observability.

Why they are studied together. The two families rest on different institutional foundations, so data become available in different ways. Family 2 has an exhaustive frame through registration under the Payment Services Act; family 5 has per-firm filings published under the disclosure obligation of the Employment Security Act. The former supplies a comprehensive frame, the latter individual variables.

Looking at one family alone, when verification fails to progress one cannot tell whether to attribute it to a defect in the frame or a defect in the variables. Setting side by side two families with different foundations makes it possible to separate which side is stuck. That is a further reason for taking both.

11.7 Variables to record

For each $\mathrm{\Phi }$, at minimum the following are recorded.

11.8 What must be pre-registered

Following the conclusions of Part III, the following are fixed and recorded before the survey begins.

(1)

The definition of the population (reference date, geographic scope, industry scope, size floor)

(2)

The threshold for treating a state as steady (the criterion for stability of $r$)

(3)

How failure cases are collected, and the range in which they cannot be

(4)

Which of the APC effects is assumed zero, and on what grounds

(5)

The criterion distinguishing primary sources from retrospective accounts

Deciding these after the fact lets the choices of Sections 12.2 and 12.3 contaminate the conclusions.