Category: systems

The fog of war in Uvalde

The interim report on the shooting in Uvalde, Texas faults the responders with treating the incident as “barricaded subject” scenario, where they should have treated it as an “active shooter” scenario.

Here are some excerpts from the report (emphasis mine)

Instead of continuing to act as if they were addressing a barricaded subject scenario in which responders had time on their side, they should have reassessed the scenario as one involving an active shooter. Correcting this error should have sparked greater urgency to immediately breach the classroom by any possible means, to subdue the attacker, and to deliver immediate aid to surviving victims. Recognition of an active shooter scenario also should have prompted responders to prioritize the rescue of innocent victims over the precious time wasted in a search for door keys and shields to enhance the safety of law enforcement responders.

Interim report, p8

An offsite overall incident commander who properly categorized the crisis as an active shooter scenario should have urged using other secondary means to breach the classroom, such as using a sledgehammer as suggested in active shooter training or entering through the exterior windows.

Interim report, p8

Although the encounter had begun as an “active shooter” scenario, Chief Arredondo testified that he immediately began to think of the attacker as being “cornered” and the situation as being one of a “barricaded subject” where his priority was to protect people in the other classrooms from being victimized by the attacker

Interim report, p52

Here’s how Chief Pete Arredondo described his mental model of the situation in the moment:

We have this guy cornered. We have a group of officers on … the north side, a group of officers on the south side, and we have children now that we know in these other rooms. My thought was: We’re a barrier; get these kids out — not the hallway, because the bullets are flying through the walls, but get them out the wall – out the windows, because I know, on the outside, it’s brick.

***
[T]o me … once he’s … in a room, you know, to me, he’s barricaded in a room. Our thought was: “If he comes out, you know, you eliminate the threat,” correct? And just the thought of other children being in other classrooms, my thought was: “We can’t let him come back out. If he comes back out, we take him out, or we eliminate the threat. Let’s get these children out.”

It goes back to the categorizing. … I couldn’t tell you when — if there was any different kind of categorizing. I just knew that he was cornered. And my thought was: “ … We’re a wall for these kids.” That’s the way I looked at it. “We’re a wall for these kids. We’re not going to let him get to these kids in these classrooms” where … we saw the children.

…

[W]hen there’s a threat … you have to visibly be able to see the threat. You have to have a target before you engage your firearm. That was just something that’s gone through my head a million times … .[G]etting fired at through the wall … coming from a blind wall, I had no idea what was on the other side of that wall. But … you eliminate the threat when you could see it. … I never saw a threat. I never got to … physically see the threat or the shooter.

Interim report, pp 52 –53

The report goes on to say:

Chief Arredondo’s testimony about his immediate perception of the circumstances is consistent with that of the other responders to the extent they uniformly testified that they were unaware of what was taking place behind the doors of Rooms 111 and 112. They obviously were in a school building, during school hours, and the attacker had fired a large number of rounds from inside those rooms. But the responders testified that they heard no screams or cries from within the rooms, and they did not know whether anyone was trapped inside needing rescue or medical attention. Not seeing any injured students during their initial foray into the hallway, Sgt. Coronado testified that he thought that it was probably a “bailout” situation.

Chief Arredondo and other officers contended they were justified in treating the attacker as a “barricaded subject” rather than an “active shooter” because of lack of visual confirmation of injuries or other information.

Interim report, p53

(Aside: A “bailout” situation refers to human traffickers who try to outrun the police. They commonly crash their vehicles and then flee. These bailout situations were so common in Uvalde that they led to alert fatigue(!). See p6 of the report for more details).

Of course, it’s impossible to know the true state of mind of the officers at the time. And, as the report notes, video camera evidence suggests that officers eventually believed there were people who had been injured by the shooter:

For example, later in the incident, Sgt. Coronado’s body-worn camera footage recorded that somebody asked, at 12:34 p.m., “we don’t know if he has anyone in the room with him, do we?” Chief Arredondo responded, “I think he does. There’s probably some casualties.” Sgt. Coronado agreed, saying “yeah, he does … casualties.” Then at 12:41 p.m.: “Just so you understand, we think there are some injuries in there.”

Interim report, p54, footnote 164

But even the report suggests that the issue was around fixation, as opposed to the officer lying about what he believed in the moment.

This “barricaded subject” approach never changed over the course of the incident despite evidence that Chief Arredondo’s perspective evolved to a later understanding that fatalities and injuries within the classrooms were a very strong probability.

Interim report, pp 53–54

My claim here is that we should assume the officer is telling the truth and was acting reasonably if we want to understand how these types of failure modes can happen.

Instead of assuming that Chief Arredondo made a mistake, if we assume he came to a reasonable conclusion in assuming the shooter was a “barricaded subject”, then we can better appreciate the ambiguous nature of incidents in the moment. In order to understand the challenges that people like Chief Arredondo faced, we need to put ourselves in his place, and imagine what our understanding would be like if we only saw the signals that he did.

This isn’t the last time a responder is going to reach the wrong conclusion based on partial information, and then get fixated on it. If we simply label Chief Arredondo as “acting unreasonably” or “being a coward”, then we might feel better when he gets fired, but we won’t get better at these sorts of failure modes. We must assume that a person can act reasonably and still come to the wrong conclusion in order to make progress.

What’s allowed to count as a cause: ALERRT edition

The Advanced Law Enforcement Rapid Response Training (ALERRT) Center, based at Texas State University, trains law enforcement officers on how to deal with active shooter incidents. After the shooting at Uvalde, ALERRT produced an after-action report titled Robb Elementary School Attack Response Assessment and Recommendations.

The “Tactical Assessment” section of the report criticizes the action of the responding officers. It’s too long to excerpt in this post, but here are some examples:

A reasonable officer would have considered this an active situation and devised a plan to address the suspect. Even if the suspect was no longer firing his weapon, his presence and prior actions were preventing officers from accessing victims in the classroom to render medical aid (ALERRT & FBI, 2020, p. 2-17).

ALERRT report, p17

In a hostage/barricade, officers are taught to utilize the 5 Cs (Contain, Control, Communicate, Call SWAT, Create a Plan; ALERRT & FBI, 2020, pp. 2-17 to 2-19). In this instance, the suspect was contained in rooms 111 and 112. The officers established control in that they slowed down the assault. However, the officers did not establish communication with the suspect. The UCISD PD Chief did request SWAT/tactical teams. SWAT was called, but it takes time for the operators to arrive on scene. In the meantime, it is imperative that an immediate action plan is created. This plan is used if active violence occurs. It appears that the officers did not create an immediate action plan.

ALERRT report, p17

(Note: per the interim report, the officers did try to establish communication with the suspect, but the ALERRT authors weren’t aware of this at the time).

At 11:40:58, the suspect fired one shot. At 11:44:00, the suspect fired another shot, and finally, at 12:21:08, the suspect fired 4 more shots. During each of these instances, the situation had gone active, and the immediate action plan should have been triggered because it was reasonable to believe that people were being killed.

ALERRT report, p18

Additionally, we have noted in this report that it does not appear that effective incident command was established during this event. The lack of effective command likely impaired both the Stop the Killing and Stop the Dying parts of the response.

ALERT report, p19

The interim report also covers some of this territory in the subsection titled “ALERRT Standard for Active Shooter Training”, which starts on p17.

What struck me after reading the ALERRT report is that there is no mention of the fact that several of the responding police officers had received ALERRT training, including the chief of the Uvalde school district police, Pete Arredondo. From the interim report:

Before joining the Uvalde CISD Police Department, Chief Arredondo received active shooter training from the ALERRT Center, which the FBI has recognized as “the National Standard in Active Shooter Response Training.” Every school district peace officer in Texas must be trained on how to respond in active shooter scenarios. Not all of them get ALERRT training, but Chief Arredondo and other responders at Robb Elementary did.

Interim report, pp 17–18

The ALERRT report discusses how the actions of the officers is contrary to ALERRT training, and that is one potential explanation for why things went badly. But another potential explanation is that the ALERRT training wasn’t good enough to prepare the officers to deal with this situation. For example, perhaps the training doesn’t go into enough detail about the danger of fixation, where Chief Arredondo focused on trying to get a key for the door, when it wasn’t even clear whether the door was locked or not. (Does ALERRT train peace officers to diagnose fixation in other responders?)

The interim report gestures in the direction of ALERRT training being inadequate when it comes to checking the locks, although not in about the more general problem of fixation.

ALERRT has noted the failure to check the lock in its criticisms. See ALERRT, Robb Elementary School Attack Response Assessment and Recommendations at 18-19 (July 6, 2022). A representative of ALERRT testified before the Committee that the “first rule of breaching” is to check the lock. See Testimony of John Curnutt, ALERRT (July 11, 2022). Unfortunately, ALERRT apparently has neglected to include that “first rule of breaching” in its active- shooter training materials, which includes modules entitled “Closed and Locked Interior Doors” and “Entering Locked Buildings Quickly, Discreetly, and Safely.” See Federal Bureau of Investigation & ALERRT, Active Shooter Response – Level 1, at STU 3-8 – 3-10, 4-20 – 4-25.

Interim report, p64, footnote 206

Now, these criticisms are hindsight-laden, and my goal here isn’t to criticize ALERRT’s training: this isn’t my domain, and I don’t pretend to know how to train officers to deal with active shooter scenarios. Rather, my point is that the folks writing the ALERRT report were never going to consider that their own training is inadequate. After all, they’re the experts!

ALERRT was recognized as the national standard in active shooter response training by the FBI in 2013. ALERRT’s excellence in training was recognized in 2016 with a Congressional Achievement Award.

More than 200,000 state, local, and tribal first responders (over 140,000 law enforcement) from all 50 states, the District of Columbia, and U.S. territories have received ALERRT training over the last 20 years.

ALERRT training is research based. The ALERRT research team not only evaluates the efficacy of specific response tactics (Blair & Martaindale, 2014; Blair & Martaindale, 2017; Blair, Martaindale, & Nichols, 2014; Blair, Martaindale, & Sandel, 2019; Blair, Nichols, Burns, & Curnutt, 2013;) but also has a long, established history of evaluating the outcomes of active shooter events to inform training (Martaindale, 2015; Martaindale & Blair, 2017; Martaindale, Sandel, & Blair, 2017). Specifically, ALERRT has utilized case studies of active shooter events to develop improved curriculum to better prepare first responders to respond to similar situations (Martaindale & Blair, 2019).

For these reasons, ALERRT staff will draw on 20 years of experience training first responders and researching best practices to fulfill the Texas DPS request and objectively evaluate the law enforcement response to the May 24, 2022, attack at Robb Elementary School.

ALERRT report, p1

I think it’s literally inconceivable for the ALERRT staff to consider the inadequacy of their own training curriculum as being a contributor to the incident. It’s a great example of something that isn’t allowed to count as a cause.

I’ll end this blog post with some shade that the interim report threw on the ALERRT report.

The recent ALERRT report states that “[o]nce the officers retreated, they should have quickly made a plan to stop the attacker and gain access to the wounded,” noting “[t]here were several possible plans that could have been implemented.” “Perhaps the simplest plan,” according to ALERRT, “would have been to push the team back down the hallway and attempt to control the classrooms from the windows in the doors.” The report explains the purported simplicity of the plan by noting: “Any officer wearing rifle-rated body armor (e.g., plates) would have assumed the lead as they had an additional level of protection.” ALERRT, Robb Elementary School Attack Response Assessment and Recommendations (July 6, 2022). A problem with ALERRT’s depiction of its “simplest plan” is that no officer present was wearing “rifle-rated body armor (e.g., plates).” The Committee agrees the officers should have attempted to breach the classrooms even without armor, but it is inflammatory and misleading to release to the public a report describing “plans that could have been implemented” that assume the presence of protective equipment that the officers did not have.

Interim Report, pp51–52, footnote 158

Uvalde

Last week, the Investigative Committee on the Robb Elementary shooting (in Uvalde, Texas) released an interim report with their findings. I recommend reading it if you’re interested in incidents, especially section 5, “May 24 Incident & Law Enforcement Response“, which goes into detail on how the police responded.

I was pleasantly surprised to see terms like contributing factors and systemic failures in the report, and not a single reference to root cause. On the other hand, there’s way too much counterfactual reasoning in the report in my taste: there’s an entire subsection with the title “What Didn’t Happen in Those 73 Minutes?” It doesn’t get more counterfactual-y than that. There’s also normative language like egregious poor decision making. It’s disappointing, but not surprising, to see this type of language, given the nature of the incident.

Reading the report, I found there was too much I wanted to comment on to fit into one post, and so I’m going to try and write a series of posts instead. I’ve also created a GitHub repo with pointers to various artifacts related to the shooting (reports, images, videos).

Imagine there’s no human error…

When an incident happens, one of the causes is invariably identified as human error: somebody along the way made a mistake, did something they shouldn’t have done. For example: that engineer shouldn’t have done that clearly risky deployment and then walked away without babysitting it. Labeling an action as human error is an unfortunately effective way at ending an investigation (root cause: human error).

Some folks try to make progress on the current status quo by arguing that, since human error is inevitable (people make mistakes!), it should be the beginning of the investigation, rather than the end. I respect this approach, but I’m going to take a more extreme view here: we can gain insight into how incidents happen, even those that involve operator actions as contributing factors, without reference to human error at all.

Since we human beings are physical beings, you can think of us as machines. Specifically, we are machines that make decisions and take action based on those decisions. Now, imagine that every decision we make involves our brain trying to maximize a function: when provided with a set of options, it picks the one that has the largest value. Let’s call this function g, for goodness.

Pushing button A has a higher value of g than pushing button B.

(The neuroscientist Karl Friston has actually proposes something similar as a theory: organisms make decisions to minimize model surprise, a construct that Friston calls free energy).

In this (admittedly simplistic) model of human behavior, all decision making is based on an evaluation of g. Each person’s g will vary based on their personal history and based on their current context: what they currently see and hear, as well as other factors such as time pressure and conflicting goals. “History” here is very broad, as g will vary based not only on what you’ve learned in the past, but also on physiological factors like how much sleep you had last night and what you ate for breakfast.

Under this paradigm, if one of the contributing factors in an incident was the user pushing “A” instead of “B”, we ask “how did the operator’s g function score a higher value for pushing A over B”? There’s no concept of “error” in this model. Instead, we can explore the individual’s context and history to get a better understanding of how their g function valued A over B. We accomplish this by talking to them.

I think the model above is much more fruitful than the one where we identify errors or mistakes. In this model, we have to confront the context and the history that a person was exposed to, because those are the factors that determine how decisions get made.

The idea of human error is a hard thing to shake. But I think we’d be better off if we abandoned it entirely.

Some additional reading on the idea of human error:

We value possession of experience, but not its acquisition

Imagine you’re being interviewed for a software engineering position, and the interviewer asks you: “Can you provide me with a list of the work items that you would do if you were hired here?” This is how the action item approach to incident retrospectives feels to me.

We don’t hire people based on their ability to come up with a set of work items. We’re hiring them for their judgment, their ability to make good engineering decisions and tradeoffs based on the problems that they will encounter at the company. In the interview process, we try to assess their expertise, which we assume they have developed based on their previous work experience.

Incidents provide us with excellent learning opportunities because they confront us with surprises. If we examine an incident in detail, we can learn something about our system behavior that we didn’t know before.

Yet, while we recognize the value of experienced candidates when we do hiring, we don’t seem to recognize the value of increasing the experience of our current employees. Incidents are a visceral type of experience, and reflecting on these sorts of experiences is what increases our expertise. But you have to reflect on them to maximize the value, and you have to share this information out to the organization so that it isn’t just the incident responders that can benefit from the experience.

To me, learning from incidents is about increasing the expertise of an organization by reflecting on and sharing out the experiences of surprising operational events. Action items are a dime a dozen. What I care about is improving the organization’s ability to engineer software.

Software engineering in-the-large: the coordination challenge

Back when I was an engineering student, I wanted to know “How do the big companies develop software? How does it happen in the real world?”

Now that I work at a company that has to do large-scale software development, I understand better why it’s not something you can really teach effectively in a university setting. It’s not that companies doing large-scale software development are somehow better at writing software than companies that work on smaller-scale software projects. It’s that large-scale projects face challenges that small-scale projects don’t.

The biggest challenge at large-scale is coordination. My employer provides a single service, which means that, in theory, any project that anyone is working on inside of the company could potentially impact what anybody else is working on. In my specific case, I work on delivery tools, so we might be called upon to support some new delivery workflow.

You can take a top-down command-and-control style approach to the problem, by having the people at the top attempting to filter all of the information to just what they need, and them coordinating everyone hierarchically. However, this structure isn’t effective in dynamic environments: as the facts on the ground change, it takes too long for information to work its way up the hierarchy, adapt, and then change the orders downwards.

You can take a bottoms-up approach to the problem where you have a collection of teams that work autonomously. But the challenge there is getting them aligned. In theory, you hire people with good judgment, and provide them with the right context. But the problem is that there’s too much context! You can’t just firehose all of the available information to everyone, that doesn’t scale: everyone will spend all of their time reading docs. How do you get the information into the heads of the people that need it? becomes the grand challenge in this context.

It’s hard to convey the nature of this problem in a university classroom, if you’ve never worked in a setting like this before. The flurry of memos, planning documents, the misunderstandings, the sync meetings, the work towards alignment, the “One X” initiatives, these are all things that I had to experience viscerally, on a first-hand basis, to really get a sense of the nature of the problem.

OR, DevOps, and LFI

I’m currently reading Systems, Experts, and Computers: the Systems Approach in Management and Engineering, World War II and After. The book is really more of a collection of papers, each written by a different author. This post is about the second chapter, The Adoption of Operations Research in the United States During World War II, written by Erik Rau.

During World War II, the British and the Americans were actively investing in developing radar technology in support of the war effort, with applications such as radar-based air defense. It turned out that developing the technology itself wasn’t enough: the new tech had to be deployed and operated effectively in the field to actually serve its purpose. Operating these systems required coordination between machines, human operators, and the associated institutions.

The British sent out scientists and engineers into the field to study and improve how these new systems were used. To describe this type of work, the physicist Albert Rowe coined the term operational research (OR), to contrast it with developmental research.

After the surprise attack on Pearl Harbor, the U.S. Secretary of War tapped the British radar pioneer Robert Wattson-Watt to lead a study on the state of American air defense systems. In the report, Wattson-Watt describe U.S. air defense as “insufficient organization applied to technically inadequate equipment used in exceptionally difficult conditions“. The report suggested the adoption of the successful British technology and techniques, which included OR.

At this time, the organization responsible for developmental research into weapons systems was the Office of Scientific Research and Development (OSRD), headed by Vannevar Bush. While a research function like OR seemed like it should belong under OSRD, there was a problem: Bush didn’t want it there. He wanted to protect the scientific development work of his organization from political interference by the military, and so he sought to explicitly maintain a boundary between the scientists and engineers that were developing the technology, and the military that was operating it.

[The National Defense Research Committee] is concerned with the development of equipment for military use, whereas these [OR] groups are concerned with the analysis of its performance, and the two points of view do not, I believe, often mix to advantage.

Vannevar Bush, quoted on p69

In the end, the demand for OR was too great, and Bush relented, creating the Office of Field Service within the OSRD.

Two things struck me reading this chapter. The first was that operational research was a kind of proto-DevOps, a recognition of the need to create a cultural shift in how development and operations work related to each other, and the importance of feedback between the two groups. It was fascinating to see the resistance to it from Bush. He wasn’t opposed to OR itself, he was opposed to unwanted government influence, which drove his efforts to keep development and operations separate.

The second thing that struck me was this idea of OR being doing research on operations. I had always thought of OR as being about things like logistics, basically graph theory problems addressed by faculty who work in business schools instead of computer science departments. But, here, OR was sending researchers into the field to study how operations was done in order to help improve development and operations. This reminded me very much of the aims of the learning from incidents (LFI) movement.

Prussia meets Versailles: a review of Moral Mazes

Managers rarely speak of objective criteria for achieving success because once certain crucial points in one’s career are passed, success and failure seem to have little to do with one’s accomplishments.

p44

Almost all management books are prescriptive: they’re self-help books for managers. Moral Mazes is a very different kind of management book. Where most management books are written by management gurus, this one is written by a sociologist. This book is the result of a sociological study that the author conducted at three U.S. companies in the 1980s: a large textile firm, a chemical company, and a large public relations agency. He was interested in understanding the ethical decision-making process of American managers. And the picture he paints is a bleak one.

American corporations are organized into what Jackall calls patrimonial bureaucracies. Like the Prussian state, a U.S. company is organized as a hierarchy, with a set of bureaucratic rules that binds all of the employees. However, like a monarchy, people are loyal to individuals rather than offices. Effectively, it is a system of patronage, where leadership doles out privileges. Like in the court of King Louis XIV, factions within the organization jockey to gain favor.

With the exception of the CEO, all of the managers are involved in both establishing the rules of the game, and are bound by the rules. But, because the personalities of leadership play a strong role, and because leadership often changes over time, the norms are always contingent. When the winds change, the standards of behavior can change as well.

Managers are also in a tough spot because they largely don’t have control over the outcomes on which they are supposed to be judged. They are typically held responsible for hitting their numbers, but luck and timing play an enormous role over whether they are able to actually meet their objectives. As a result, managers are in a constant state of anxiety, since they are forever subject to the whims of fate. Failure here is socially defined, and the worst outcome is to be in the wrong place at the wrong time, and to have your boss say “you failed”.

Managers therefore focus on what they can control, which is the image they project. They put in a lot of hours, so they appear to be working hard. They strive to be seen as someone who is a team player, who behaves predictably and makes other managers feel comfortable. To stand out from their peers, they have to have the right style: the ability to relate to other people, to sell ideas, appear in command. To succeed in this environment, a manager needs social capital as well as the ability to adapt quickly as the environment changes.

Managers commonly struggle with decision making. Because the norms of behavior are socially defined, and because these norms change over time, they are forever looking to their peers to identify what the current norms are. Compounding the problem is the tempo of management work: because a manager’s daily schedule is typically filled with meetings and interrupts, with only fragmented views of problems being presented, there is little opportunity to gain a full view of problems and reflect deeply on them.

Making decisions is dangerous, and managers will avoid it when possible, even if this costs the organization in the long run. Jackall tells an anecdote about a large, old battery at a plant. The managers did not want to be on the hook for the decision to replace it, and so problems with it were patched up. Eventually, it failed completely, and the resulting cost to replace it and to deal with costs related to EPA violations and lawsuits was over $100M in 1979 dollars. And yet, this was still rational decision-making on behalf of the managers, because it was a risk for them in the short-term to make the decision to replace the battery.

Ethical decision making is particularly fraught here. Leadership wants success without wanting to be bothered with the messy details of how that success is achieved: a successful middle manager shields leadership from the details. Managers don’t have a professional ethic in the way that, say, doctors or lawyers do. Ethical guidelines are situational, they vary based on changing relationships. Expediency is a virtue, and a good manager is one who is pragmatic about decision making.

All moral issues are transmuted into practical concerns. Arguing based on morality rather than pragmatism is frowned upon, because moral arguments compel managers to act, and they need to be able to take stock of the social environment in order to judge whether a decision would be appropriate. Effective managers use social cues to help make decisions. They conform to what Jackall calls institutional logic: the ever-changing set of rules and incentives that the culture creates and re-creates to keep people’s perspectives and behaviors consistent and predictable.

There comes a time in every engineer’s career when you ask yourself, “do I want to go into management?” I’ve flirted with the idea in the past, but ultimately came down on the “no” side. After reading Moral Mazes, I’m more confident than ever that I made the right decision.

Taking the hit

Here’s a scenario I frequently encounter: I’m working on writing up an incident or an OOPS. I’ve already interviewed key experts on the system, and based on those interviews, I understand the implementation details well enough to explain the failure mode in writing.

But, when I go to write down the explanation, I discover that I don’t actually have a good understanding of all of the relevant details. I could go back and ask clarifying questions, but I worry that I’ll have to do this multiple times, and I want to avoid taking up too much of other people’s time.

I’m now faced with a choice when describing the failure mode. I can either:

(a) Be intentionally vague about the parts that I don’t understand well.

(b) Make my best guess about the implementation details for the parts I’m not sure about.

Whenever I go with option (b), I always get some of the details incorrect. This becomes painfully clear to me when I show a draft to the key experts, and they tell me straight-out, “Lorin, this section is wrong.”

I call choosing option (b) taking the hit because, well, I hate the feeling of being wrong about something. However, I always try to go with this approach because this maximizes both my own learning and (hopefully) the learning of the readers. I take the hit. When you know that your work will be reviewed by an expert, it’s better to be clear and wrong than vague.