Interpreting pigeons' standard output

During the execution of parallel tempering, interim diagnostics can be computed and printed to standard out at the end of every iteration (this can be disabled using show_report = false):

using Pigeons

pigeons(target = toy_mvn_target(100))

┌ Info: Neither traces, disk, nor online recorders included.
│    You may not have access to your samples (unless you are using a custom recorder, or maybe you just want log(Z)).
└    To add recorders, use e.g. pigeons(target = ..., record = [traces; record_default()])
────────────────────────────────────────────────────────────────────────────
  scans        Λ        time(s)    allc(B)  log(Z₁/Z₀)   min(α)     mean(α)
────────── ────────── ────────── ────────── ────────── ────────── ──────────
        2        7.5   5.23e-05   1.28e+04       -122   2.66e-15      0.167
        4       5.59   8.69e-05   1.47e+04       -119    2.6e-07      0.378
        8       6.04   0.000109   1.86e+04       -115    0.00129      0.329
       16       7.27   0.000162   2.42e+04       -118     0.0134      0.193
       32       6.97   0.000282   3.05e+04       -114      0.107      0.225
       64       7.03   0.000467   4.13e+04       -117     0.0531      0.219
      128       7.23   0.000877   6.08e+04       -114     0.0944      0.196
      256       7.05    0.00156   6.77e+04       -115       0.13      0.217
      512       7.14    0.00292   7.18e+04       -115      0.171      0.207
 1.02e+03       7.19    0.00565   7.91e+04       -115      0.172      0.201
────────────────────────────────────────────────────────────────────────────

The functions called to emit each of these can be found at all_reports(). Some key quantities:

• Λ: the global communication barrier, as described in Syed et al., 2021 and estimated using the sum of rejection estimator analyzed in the same reference. Syed et al., 2021 also developed a rule of thumb to configure the number of chains: PT should be set to roughly 2Λ.
• time and allc: the time (in second) and allocation (in bytes) used in each round.
• log(Z₁/Z₀): the stepping_stone() estimator for the log of the normalization constant, see the documentation page on approximation of the normalization constant.
• min(α) and mean(α): minimum and average swap acceptance rates over the PT chains.

Additional statistics can be shown when more recorders are added. For example, to accumulate other constant-memory summary statistics:

pigeons(target = toy_mvn_target(100), record = record_online(), explorer = AutoMALA())

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
  scans     restarts      Λ        time(s)    allc(B)  log(Z₁/Z₀)   min(α)     mean(α)    max|ρ|     mean|ρ|    min(αₑ)   mean(αₑ)
────────── ────────── ────────── ────────── ────────── ────────── ────────── ────────── ────────── ────────── ────────── ──────────
        2          0        6.2      0.189   1.04e+07       -131   3.85e-13      0.311          1          1      0.284      0.536
        4          0       6.18    0.00217   8.58e+04       -123   7.56e-05      0.314      0.827       0.37       0.41      0.535
        8          0       6.91    0.00382   9.47e+04       -115     0.0183      0.232      0.637      0.502      0.438      0.567
       16          0       7.14     0.0077   1.49e+05       -114     0.0311      0.207      0.547      0.296      0.476      0.537
       32          0       6.66     0.0149   1.93e+05       -116     0.0441       0.26      0.514      0.279      0.485      0.539
       64          0       6.79     0.0298   2.82e+05       -116     0.0963      0.246      0.617      0.334      0.463      0.535
      128          0       7.27     0.0589   4.08e+05       -115      0.126      0.192      0.357      0.295      0.451      0.533
      256          2       7.22      0.118   6.91e+05       -114      0.136      0.198      0.444      0.307      0.475      0.533
      512          4       7.19      0.234   9.19e+05       -116       0.15      0.202      0.408      0.302      0.485      0.531
 1.02e+03         12       7.21      0.467   1.49e+06       -115      0.167      0.199      0.428       0.33      0.483      0.531
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

• max|ρ| and mean|ρ|: maximum and average (across chains) correlation of the random variables $L^t_i = V(X_i)$ and $L^{t+1}_i = V(X_i)$ where $V = \log \pi_N / \pi_1$, $X_i \sim \pi_{\beta_i}$, and $t, t+1$ are indices just before and after a call to step!().
• min(αₑ) and mean(αₑ): minimum and average (across chains) of the explorer's acceptance rates.

Programmatic access

The tables described above can also be accessed as a DataFrame via:

using Pigeons
pt = pigeons(target = toy_mvn_target(100))
pt.shared.reports.summary

Detailed statistics can be accessed via these DataFrames. For example to obtain mean swap acceptance probabilities for each round and pair of communicating chains, use:

first(pt.shared.reports.swap_prs, 20)

Creating a plot from this:

using StatsPlots
plotlyjs()
my_plot =  @df pt.shared.reports.swap_prs StatsPlots.plot(:round, :mean, group = :first)
StatsPlots.savefig(my_plot, "swap_prs.html");