Wednesday, July 20, 2011

Understanding Trust

It's often been said that Confidentiality, Integrity, and Availability, the so-called CIA triad, are the core principles of information security. However, I want to examine even something more fundamental than these principles; I want to look at the goals of information security. And not just goals such as preventing unauthorized access, disclosure, disruption of use, etc. which really are just extensions of the CIA triad, but the core, essential goals that are at information security's foundation.

At its core, information security is largely about the two goals of “ensuring trust” and “managing risk”. We may deal with managing risk some other time, but today I want to focus on ensuring trust.

In order to ensure trust, we first must understand not only what it is, but what its properties are.

Let's start with a definition. Merriam-Webster's dictionary defines the noun, trust as:
1 a : assured reliance on the character, ability, strength, or truth of someone or something b : one in which confidence is placed
2 a : dependence on something future or contingent : hope b : reliance on future payment for property (as merchandise) delivered : credit <bought furniture on trust>
3 a : a property interest held by one person for the benefit of another b : a combination of firms or corporations formed by a legal agreement; especially : one that reduces or threatens to reduce competition
4 archaic : trustworthiness
5 a (1) : a charge or duty imposed in faith or confidence or as a condition of some relationship (2) : something committed or entrusted to one to be used or cared for in the interest of another b : responsible charge or office c : care, custody <the child committed to her trust>

One thing that I'd like to draw your attention to is that none of these definitions (with the possible exception of #3) implies that any sort of qualitative measure for trust exists. Trust is one of those things that we think that we completely understand when we talk about it, but when we explore it a bit deeper, we discover that it has some properties that are not all that intuitive, at least not in the normal manner that we refer to trust. My intent is to help us gaze at some of these properties and in so doing, see the disconnect of how we use the word “trust” in everyday usage versus how we use it in the security world. As we will discover, it is some of these very properties that make trust so difficult to ensure in the world of information security.

Properties of Trust

Trust is not commutative

If Alice trusts Bob, it does not follow that Bob trusts Alice. To any parent, this seems pretty obvious, at least when your children are of a certain age where they still trust you. You trust you doctor, but they likely do not trust you in a similar manner. You trust your bank, but they don't trust you in the same way. Trust is not symmetrical in this way.

Trust is transitive

This means “If Alice trusts Bob, and Bob trusts Carol, then Alice trusts Carol”. This one doesn't seem quite so obvious to us. That's because in the real world where we interact with other people, we don't treat trust in this manner. If I trust my wife and my wife trusts her friend, I don't usually automatically extend my trust to her friend. But therein lies the problem. In the information security context at least, trust implicitly extends in this way.

Think of the analogy where Alice, Bob, and Carol are all names of servers and there are trust relationships established by virtue of a /etc/hosts.equiv on each of these servers. When described this way, it is easier to see how trust extends to be transitive even though Alice may not be aware of Bob's trust of Carol. (In a latter blog post, I hope to illustrate how this affects trust boundaries.) Alice (usually implicitly) extends here trust to Carol via Bob's trusting Carol. The big issue here is whether or not Alice's trust of Bob is warranted in the first place and that is muddied by the fact that rational humans (at least those who are worthy of that trust) will act in a moral way so as not to abuse it. But in computer security, we need to think beyond this. The big issue in making such trust decisions involving transitivity is that Alice may not be aware of any of the trust relationships that Bob has with any other parties. For example, if Alice knows Carol to be untrustworthy, she may rethink her position on trusting Bob. In other words, Alice's trust in Bob may be misplaced. The bottom line is that there's little that Alice can do to tell, except to go on Bob's known reputation. Where this really gets sticky of course is that it can extend indefinitely. Alice trusts Bob, Bob trusts Carol, Carol trusts David, David trusts Emily, so... transitivity implies that Alice trusts Emily. We can quickly get lost. Of all the aspects of trust, I think that it is this disconnect that humans have with trust being transitive that makes securing computers so difficult. We simply don't think this way intuitively when it comes to securing our systems.
Let's try to clarify this with an example. Let's say that you (Alice) trust a merchant (Bob's Bakery) with your credit card number. You do this by paying for your weekly doughnut supply with your credit card (online or in person, it really doesn't matter). Bob's Bakery relies on a credit agency (Carol's Credit Check) to do credit checks on its customer's credit cards before accepting each payment.
No problem so far, right? Now you (Alice) may or may not be aware that Bob's Bakery does a credit check. As long as they accept your payment and you get your sweets, you probably don't care. You probably are not consciously thinking about any credit agency or credit card payment service. Most people are not even aware of the connection. Regardless, you are implicitly extending your trust to this credit agency when you you complete such a transaction.
No problem, still, you say. My credit card issuer will cover any fraudulent charges. But note that is a red herring. That is why you trust your bank and your credit card, not why you trust Bob's Bakery or Carol's Credit Check.
So let's change up the scenario a bit. Let's assume that Carol's Credit Check is really run by organized crime and the purpose of their existence is to perpetuate credit card fraud. Does this change your trust of Carol's Credit Check (assuming you new of the relationship with Bob's Bakery)? Probably. Does it change your trust in Bob's Bakery? It may, if Bob's Bakery were aware that Carol's Credit Check was run by crooks. But why does you trust change? Nothing has changed except your awareness / perception. In abstract terms, it's still the same Alice trusts Bob, Bob trusts Carol, therefore Alice trusts Carol.

Trust is not binary

Trust is a matter of degree; it is gray scale, not something that is merely black or white, on or off. If this were not true, there would be no way for us to state that we trust one party more than some other party. If trust were only a binary yes/no, if Alice separately trusted Bob and Carol, Alice would never has any reason to trust Bob more than she trusts Carol. Obviously that is not how we act in the real world. Often times, even when we do not personally know someone, we grant different levels of trust based solely on reputation.

Trust is context dependent

Alice may trust Bob in some situations, but not in other situations. While this is true in computer security, it is a rather difficult property for us to model. But there is no doubt that we believe this in the real world. In real life, Alice may trust Bob to cut her hair or work on her car, but not to baby sit her kids, and almost certainly not to preform brain surgery on her (even if Bob has taken the time to read Brain Surgery for Dummies).
In real life, we humans are quite adept at switching between these various contexts. So much so, that we hardly give it conscious thought. However, when we try to codify these contexts into a programmatic information model, we realize that this is a very difficult concept to formalize.

Trust is not constant over time

Alice may trust Bob at time t, but not at some later time t+Δt. In real life, Bob may do something to screw up. For instance, Alice may trust Bob while she is dating him, but after she sees Bob chasing after Carol, not so much. In computer security, we have similar analogies. For instance, we trust a session ID until after it has expired, or we trust a certificate until it has expired. It is at the basis of why we recommend that passwords expire after a time. This property of trust is one reason that authentication protocols use things like expiration times and/or freshness indicators.

Final Observations

These last two properties (trust is context dependent, trust is not constant over time) mean that when we discuss ensuring trust, we must do so within a specific context and time-frame. (Sometimes the time-frame is explicit and sometimes it is implied.) Generally, in computer security, we should strive to make all trust relationships explicit and leave nothing to chance or misinterpretation. That's one key step in defining a trust model, which I hope to discuss in a future blog post. In the meantime, I hope you will keep in mind some of the properties we discussed today when you are trying to secure your systems.

And one more thing, my apologies for not being consistent as of late posting to this blog. Not only have I been busy for with ESAPI for C++, but I've also been at loss for interesting topics. So if you have something that you'd like to see me ramble on about, please shot me a note. (But don't be surprised if I recommend that you get so serious counseling if you do so. ;-) Thanks!
-kevin