1
00:00:00,050 --> 00:00:28,183
Deep dive number eight. The Failure Study tab. The Error Study asked what many small imperfections do together. This tab asks the opposite. What happens when one thing breaks completely. A cavity trips, a magnet supply dies. For a superconducting linac like PIP two, which must keep running through cavity failures, this is the resilience question. Everything here is produced live. Ranking, heatmap, and at the end, a genuine rescue.

2
00:00:28,283 --> 00:00:56,447
The whole tab, D T L section loaded, protons at three M e V. Down the left, the questions. Which elements may fail. How they fail. Alone, in pairs, or in named sets. Whether to attempt a rescue. Plus a Run group. The column scrolls on its own, so the plots never get squeezed. On the right, the answers. Ranking table, criticality bar, and a pane reserved for the pair failure heatmap. Each will earn its keep.

3
00:00:56,547 --> 00:01:25,512
Targets first. Four type boxes, Cavity, Quad, Solenoid, Dipole, and the list of failable elements, each with its classification in brackets. Only named, active, uniquely named elements qualify. A drift cannot break, and a duplicated name is excluded, because failures are injected by name and a name matching two elements would be ambiguous. And field maps classify by content, accelerating means cavity, otherwise solenoid.

4
00:01:25,612 --> 00:01:42,646
The boxes filter the list live. Untick Quad and the eight quadrupoles leave, eighteen entries become ten. Untick Cavity and only the two solenoids remain. Tick them back and the roster returns. No dipoles in this lattice, so that box changes nothing here.

5
00:01:42,746 --> 00:01:58,664
Selection narrows the sweep. Click four elements and only those four fail. The rule sits right above the list. Select a subset, or none means all. An empty selection is the full sweep, not an empty one. Remember this trick for the pairs run later.

6
00:01:58,764 --> 00:02:24,751
The failure mode. Off is total. It rides the element's additive error slot, relative strength to minus one hundred percent, nothing transfers. Partial scales a magnet to a fraction of design. Detune keeps a cavity running but off set point, amplitude scale times nominal plus an additive phase offset in degrees. Amplitude accepts zero to two, phase spans plus or minus one hundred eighty.

7
00:02:24,851 --> 00:02:54,851
Fields wake only for modes that use them. On off, both are greyed. Detune enables both, and the amplitude flips from zero point nine to one, so a pure phase detune cannot silently scale the voltage. Partial greys the phase and resets amplitude to zero point nine, a ten percent droop. Back on detune we dial zero point nine and plus ten degrees. A run would label each scenario, detune, amp zero point nine zero, phi plus ten. Now back to off.

8
00:02:54,951 --> 00:03:24,551
Combination sets the ambition. Single fails each element alone, giving the criticality ranking. Pairs adds every unordered pair, N plus N choose two. Our eighteen elements mean one hundred seventy one scenarios, which is why the manual says run single first, then pairs on the few worst. The singles run first and fill the heatmap diagonal. Custom builds explicit failure sets, exactly the scenario your review board asked about. Let us build one.

9
00:03:24,651 --> 00:03:47,938
Select a quadrupole and its neighbouring gap, then Add selected as set. The pair lands in the set list joined by a plus, and the combination flips to custom by itself. Each set fails as one unit. A second set, the same way. Clear sets empties the list. Sets survive a reload with the same names and are pruned if their names disappear. We reset to single.

10
00:03:48,038 --> 00:04:28,767
The most interesting group. Fault recovery. Tick re tune neighbours and HELIX tries to rescue the worst cases after the sweep, MYRRHA style. Strategy picks the rescuers. K out of n, the k nearest same category elements, upstream winning ties. L neighbouring lattices, whole focusing periods. Manual, names you supply in the C L I or A P I. One spinner feeds both k and l. Algorithms, C M A E S, least squares, or Bayesian optimisation. The cost solver scores each attempt, envelope for speed, M P when transmission must be recovered, separate from the forward model. Top N caps the attempts.

11
00:04:28,867 --> 00:05:01,300
The Run group. Forward model is the sweep's physics. Envelope, the fast R M S model, tracks no particle loss, so transmission and loss will read a dash. M P tracks real particles through real apertures. Workers is deliberately greyed out. The G U I sweeps serially, in process, on the in memory lattice, so unsaved edits are honoured exactly. Parallel workers belong to the C L I. Run starts a background worker, Stop cancels between scenarios, and the status reads idle.

12
00:05:01,400 --> 00:05:31,419
Time to break things. Eighteen elements, mode off, envelope model. Run. Nineteen simulations, the healthy baseline plus eighteen failures, finish inside a second, envelope speed is the point of this model. The table lands already sorted in ranking order, worst first, a recovered column header appears for later, and the status reads done, eighteen scenarios. On a heavier sweep the table, bar and heatmap fill live, and the pairs run will show exactly that.

13
00:05:31,519 --> 00:06:12,886
The ranking tells a physical story. Two quadrupoles top the chart. Losing QUAD zero zero seven blows the normalised horizontal emittance from one point seven seven up to three point six two, more than double, outweighing any single cavity. The eight gaps rank together below, each costing the same half M e V of exit energy. And notice the honesty at the bottom. Four quadrupoles and both solenoids score at or near zero. The score counts damage, never improvement. The dashes mean not modelled, not zero, the envelope model declaring its limits. And epsilon n z is beta gamma times epsilon z, the usual convention.

14
00:06:12,986 --> 00:06:34,515
The criticality bar, worst on the left, each bar labelled with its failed element, so a bar reads straight back to a table row. It caps at the top fifteen. With eighteen singles, the three mildest never chart. On a pairs run the label joins both names with a plus. This is the review slide figure. One glance says which failure hurts most.

15
00:06:34,615 --> 00:07:05,281
Where the number comes from. A weighted sum of damage against the healthy baseline. Fractional transmission loss times ten. Fractional exit energy deviation times five. Growth of each plane's normalised emittance, times one apiece. Normalised, so a decelerated beam is not punished twice through lost adiabatic damping. Unrecorded terms contribute zero. And a scenario is flagged beam lost when transmission falls below one percent or the exit energy goes non finite.

16
00:07:05,381 --> 00:07:39,681
Now the expensive question. Pairs, on every element, forward model switched to M P, fifty thousand macro particles per scenario. One hundred seventy one scenarios, and the status line says so with a warning. Pairs of eighteen, slow. Consider envelope, a subset, or the C L I workers flag. The eighteen singles run first, so the heatmap diagonal fills before any off diagonal cell. We let it grind for a while, then Stop. The worker is interrupted between scenarios, the status reports cancelled by user, and Run comes back.

17
00:07:39,781 --> 00:08:17,015
The affordable version. Back on envelope, we select the six worst offenders from the single ranking, two quadrupoles and four gaps. Pairs on six is twenty one scenarios, and the envelope model finishes the whole matrix in about a second, same fill order as before, diagonal then pairs. The pane takes its title, pair failure criticality. The bright row and column belong to QUAD zero zero seven. And the brightest off diagonal cell is QUAD zero zero five plus QUAD zero zero seven, two point eight seven, worse than either alone. The interaction is the finding.

18
00:08:17,115 --> 00:08:44,946
Reading it. The matrix is symmetric, failing A with B is failing B with A, both cells written together. Axes, failed element i and j. The scale is inferno with its colour bar, dark benign, bright dangerous, and the diagonal is the single ranking you already know. Tick labels adapt. A shared family prefix like F MAP underscore gets stripped, and a huge matrix thins its ticks. Our six names simply fit.

19
00:08:45,046 --> 00:09:21,367
Every transmission cell so far was a dash. Time to fill them. A new venue, the halo benchmark channel, twenty four identical FODO cells, and a matched one M e V proton beam the healthy machine transports without loss. Five mid channel quadrupoles, mode off, forward model M P. Every scenario is now a real multi particle run through real apertures, and the columns fill with measured numbers. The worst, QUAD zero one two, drops transmission to eighty nine point six percent. Ten point four percent of the beam, actually lost, not just emittance grown.

20
00:09:21,467 --> 00:10:08,000
The finale. Can the machine save itself. The failure analysis demo, four cavities and three solenoids, protons at two and a half M e V. Re tune neighbours on, k out of n with k two, least squares on the envelope cost, compensate top three. Run. The sweep ranks the cavities, worst is GAP zero zero four, over one point one M e V of missing energy. Then, for each of the three worst, HELIX plants temporary ADJUST cards on the neighbours' voltage and phase, half to one and a half times amplitude, plus or minus thirty degrees, adds a set K E out min objective, and reruns the matcher. The table gains recovered columns, and the status settles at done, seven scenarios, three of three recovered.

21
00:10:08,100 --> 00:10:39,429
The verdicts up close. A check mark is a rescue the matcher achieved, design exit energy back within five hundredths of an M e V. All three succeed, each using the three surviving cavities. Rows below the top three were never attempted, so their verdict stays blank, and a cross, when you see one, is an honest no. T rec stays a dash because the cost solver was envelope. Switch it to M P when transmission itself must be recovered. The recovered case emittances sit beside the broken ones.

22
00:10:39,529 --> 00:11:09,625
The same engine drives a headless command line. List elements prints the failable roster, here filtered to the four cavities. The full run wires every option you have seen, and here Workers is real, a parallel process pool for full pairs sweeps. This transcript is genuine. Seven scenarios, the worst compensated, recovered true with the three neighbours named, and a C S V carrying every metric plus the recovered flag, compensators, and matched settings, ready for a notebook.

23
00:11:09,725 --> 00:11:38,678
Prefer scripting it. Three calls. Enumerate scenarios builds the failure list and the name to class map the injector needs. Failure study dot run sweeps them, in memory, exactly like the G U I. Compensate takes the worst scenario and a config, and returns the verdict, compensator names, and matched settings. Every name is a real export, and a runnable demo ships in examples slash failure analysis, ending on the same recovered true.

24
00:11:38,778 --> 00:12:10,411
Everything here is written down. The failure study chapter opens with the layout, then every control group. The mode table. The order N squared pairs warning. The strategies. Further down, the exact criticality weights, the C L I invocation, the A P I snippet, and three notes worth reading twice. Envelope tracks no loss, a dash is a statement. A blank heatmap usually means the sweep is still running. And recovered false is an honest answer. It lives right after the error study tab.

25
00:12:10,511 --> 00:12:37,811
That is the Failure Study tab. Break everything on purpose, alone and in pairs. Rank the damage with a score that cannot be flattered. Stop a sweep that outgrows its budget. Measure real loss with M P. And let the matcher practise the rescue before the real machine needs one. The manual carries every number we quoted. Elsewhere in the series, the Surrogates tab teaches neural networks to stand in for expensive physics. See you there.