Tuesday, March 15, 2011

Response to Mark Curphey's Blog “OWASP—Has It Reached a Tipping Point”

Back on February 18, 2011, Mark Curphey, the original founder of OWASP, wrote a very thought provoking blog regarding the direction that OWASP seemed to be taking.

And while I hadn't originally intended on commenting on it, Rohit Sethi's post to the Secure Coding mailing list, caused me to rethink this. (Pardon me for not addressing everything in the sequential order that Mark brings them up.)

Curphey's blog refers to tweets coming from the OWASP Summit in Portugal as the singular event that precipitated his reflections about OWASP's directions. He points out the tweet that caught his eye was one that where John Wilander tweeted that an OWASP Summit attendee remarked on stage that “Developers don't know shit about security”. Curphey refers to John Wilander's post “Security People vs. Developers” which Wilander blogged about 5 days after his initial Twitter post. Wilander also has a very thoughtful post. In Wilander's blog, he says he latter added the comment “Well, I got news. You don't know shit about development”, obviously in reference to the OWASP Summit attendee who originally made the comment.

Well, I'm going to respond in part to both Curphey's and Wilander's blog posts, and perhaps an a little (in)sanity to the mix. Why? Well, for one, I have extensive experience in both development and security. I've been a developer for 30+ years, and been involved with application security for about 12 years. So, IMNSHO, I know “shit” about both developer and security and it is that frame of reference that I am posting this.

So Who Is Right?

So let's start out trying to answer the question, who is correct here? Which if any of these are the correct point of view:
  • Developers don't know jack about security.
  • Security people don't know jack about development.
Or perhaps, let's go even further and contemplate:
  • Developers don't know squat about development.
  • Security professionals don't know squat about security.
Well, I could answer “none of the above” or I could just as easily answer “all of the above”. That alone should tell us that we are trying to answer the wrong question. But I have honestly seen developers (some of them even with PhDs in computer science) who couldn't write code if their life depended on it and architects who no longer remember how to code. I've also seen security professionals who are great pretenders. They know all the right buzzwords and have all the right certifications (e.g., ISC2's CISSP, GIAC's Information Security Professional, etc.), but ask them to write up a threat model for some system and all the can give you is a blank stare.
That is not the point here!
A former colleague and close friend of mine once told me that he thought that about 20-25% of the people in any profession are clueless @holes. Any while that may be true (I think that the percentage is a rather high, and am in no position to judge other professions so I will refrain from further comment), that isn't the point. Companies, professional societies, and society as a whole have to play they hand we've been dealt. Companies generally don't have the option to use another company's employees.So the real point is not “who is correct?” in this debate, but rather “how can we help IT as a whole to move into a direction of more secure code without sacrificing developer (and perhaps more importantly, business) interests?”.
Which brings us back to the original "developers don't know shit about security comment". While I completely understand one's needs occasionally to vent frustration (and a cage wrestling match between Linus Torvalds and Bruce Schneier might even prove amusing ;-), sniping at each other is seldom a productive way to reach your goals, as all it does is alienate the two parties that need very much to be working together to solve these issues. Taking an “us versus them” mentality is bound to fail, but by taking a “collective 'we'” stance, we might just make some progress.

So What Does Matter?

Well, we could start with respect, for one. Now I will be the first to admit that I do a poor job at this. All too often, I confuse the issues of “trust” vs “respect”. I believe that it is OK to require that people earn some degree of trust. It is not OK to try to make them earn respect. There should be a certain amount of respect that we have for our both our development and security colleagues just based on common dignity of humankind and the common profession to which we belong.
What is not respectful is to assume that you clearly understand the motivations of individuals. At times, we are all probably guilty of this. So we might make the assumption that all VB programmers are stupid. I must confess that I have propagated this myth by quoting Dijkstra's comment on BASIC numerous times in my email .sig and that is disrespectful. Likewise for my .sig with Richard Clarke's quote “The reason you have people breaking into your software all over the place is because your software sucks...”. So I just want to be the first to step up and admit my guilt and to try to begin the healing process. I am unfortunately rather jaded over my 30+ years in IT, seeing the same dumb things repeated over and over again, often by the same people (myself included). But that is no reason to be disrespectful to them. So if I've offended anyone, I apologize for being so inconsiderate and ask your forgiveness. And even though the Dijkstra and Clarke quotes are two of my favorites, I will not use them again in my email .sig.

Reaching Common Ground

Wilander cites a poll that he took of 200+ developers and asked them to rate where security fell in their priority. Their rankings came out like this:
  1. Functions and features as specified or envisioned
  2. Performance
  3. Usability
  4. Uptime
  5. Maintainability
  6. Security
If you have done any development in the trenches, you probably are not surprised at these things. But I think Wilander let off one important rating. When I talk to developers and indeed in my former life as one, “schedule” would always come up as #1, and not overrunning the budget was always a close second or third. Perhaps meeting schedule and budget was just implied; I don't know as I haven't see the poll to which John refers. However, if nothing else, it does point so some other contributing forces as to why security gets ranked so low. (Surprisingly, this is often even the case when security software is involved, so these forces—probably coming from the business—seem to be universal, at least in the commercial sector.)
I have often said that I believe that development is more difficult than security. Why? Well, for one, developers are expected to deal with the security aspect of their software plus all these other items in Wilander's list.
Because of that, I think that my observation is true in the general case:
    “It's easier to teach a good developer about security than it is to teach a good security person how about software development.”
So if I have to start training people about security (and I have done this with several), I always prefer to start with someone who is a good developer teach them about security rather than going in the other direction.

Back to Curphey's Blog

Which brings me back, albeit in a circuitous way, to Mr. Curphey's astute observations.
Mark states:
    “I had always hoped that the community would develop into a community of developers that were interested in security rather than a community of security people that were interested in software. I wanted to be part of a community that was driving WS* standards, deep in the guts of SAML and OAuth, framework (run-time) / language security and modern development practices like Agile and TDD rather than people seemingly obsessed by HTTP and web hacking techniques.”
Why hasn't this happened? Well, in my opinion, one reason is that an open source community's involvement in standards work is definitely at a disadvantage to special interest groups comprised of well-funded vendors who have a vested interest to develop such forward-looking work for the benefit of their respective companies. Involvement in these standards is takes a lot of time and usually there is a low reward, that being seeing people adopt your respective standard. Look how long it is taking the various OASIS WS-* standards to gain critical mass.
Secondly,this sort of ambitious work requires a broad base of expertise. Let's take SAML, for instance. That requires expertise in XML, XSDs, encryption, and authentication protocols, and experience in writing specifications is highly recommended as well. That sort of breadth, beyond a surface level, is rare in individuals. However, a company sponsoring a specific standard need not rely a a single individual (even if they may have a single point of contact); they can afford to have several people participate. After all, why not? There frequently is a profit motive there. (This is especially true in standards that on the 2.0 or later revisions.)
Thirdly, people typically stay with what they are comfortable. So if their expertise is HTTP-based attacks, they stick with it until it no longer provides their meal ticket. And let's face it, when OWASP was started a decade or so ago, HTTP was pretty much all you needed to understand to get by. (Well, that plus a fundamental understanding of JavaScript.)
Mr. Curphey continues with
    “We can’t have security people who know development. We must have developers who know security. There is a fundamental difference and it is important.”
I wholeheartedly agree with this. I think it aligns well with my above observation that it's easier to take a good developer and train her about security than vice-versa. If we fail to keep this idea in the forefront of our minds, we are bound to fail. My only addendum to Mark's comment would be to rephrase it to say that “we can't only have security people who know development”. Those folks are still valuable. I think and hope Mark would concur.
Curphey continues...
    Manage the Project Portfolio – When I look at the OWASP site today its hard to see it as anything else but a “bric-a-brac” shop of random projects. There are no doubt some absolute gems in there like ESAPI but the quality of those projects is totally undermined by projects like the Secure Web Application Framework Manifesto. When I first looked I honestly thought this project was a spoof or a joke.”
I see the same disorganization. In part, I think that's in part something that Wikis seem to encourage. Once they evolve beyond a certain critical mass, similar things get written down many times on many different pages unless there is an overall editor / project manager to manage it all. AFAIK, OWASP does not have this and it suffers for it. I think it also explains the lack of uniform consistency and quality across the various OWASP projects.
Also, while I appreciate Curphey's candor, I appreciate even more Rohit Sethi's non-defensive response. When I read Rohit's comment to Mark's blog, I must say that while I'm sure it must have been a hard pill for him to swallow, he stepped up and did so like an honorable man. However, I do disagree in part with Curphey's assessment of the OWASP “Secure Web Application Framework Manifesto”. I don't think it is a total waste. In reply posted earlier today by Benjamin Tomhave, seems to have similar sentiment when Ben writes “but I hate to see the white paper lost. Why not also look at joining efforts with something like the Rugged Manifesto movement?”. There is something to be redeemed in almost every mess. If nothing else, it is useful to examine in more detail to see why it was deemed a failure. (I must admit I only took a quick 5 minute read through the whole manifesto, however, my overall sense was not that it lacked valuable information, but rather that it lacked appropriate organization along with a certain degree of incompleteness. If it were written in such a way that could be referenced BY DEVELOPERS (your audience) as a specification document, it could prove useful...especially for new frameworks that are only now getting underway. But if such a document is not well-organized and approachable from the point of view of developers, it will not get used. Period! (And don't even ask why 'Period' demands an exclamation point; I just like the sense of irony.)
Regarding Curphey's second point on “Industry Engagement and Communications”, I have no personal basis from which to speak, so I'll just keep my mouth shut on this one. (I can hear you all saying “Thank God!” :)
To Mark's third point on “Ethics / Code of Conduct”, I think he is spot-on. Vendors use the “OWASP” word to sell the products more than ever. (“Successfully defends against the OWASP Top Ten attacks”, etc.) But unless OWASP is willing to take a stand and bite the hand that sometimes feeds it (Mark's second point), this will never change. In particular, unless OWASP is willing to litigate against those who take advantage of the OWASP name to pander their products, I don't see this changing at all. IANAL and thankfully, don't even play one on TV (or YouTube or that matter.), so that is up to the OWASP Board to decide, not me. (Hey, Jeff! Are you listening? What's your $.02 on this?)
And finally to Mark's last point “Engaging Developers”. Mark writes:
    “Maybe Software Security is for developers and Application Security is for security people. The first persona is the builder and the second persona the breaker. ... Developers best understand what they need and want, security people best understand what they need and want.”
No! I don't think so. There may be a continuous spectrum from builder to breaker, but if we treat these as independent goals, we will never get to where I think we all want to be, which is “secure software”. So they better have a common goal; they better both “need and want” the same things. Otherwise the whole security effort overall will fragment and fall apart. We need each other, and breakers and builders do have different means to achieve the same goal. But it had better be the same goal (well, assuming you breakers are “white hats”), and that goal is to improve software and systems security. Don't forget that!

Wow; I've blathered on much longer than intended. My original intent was to post this as a reply to SC-L, but at 4 pages, its a bit long for that. (Well, if you only learned one thing from this post, it's probably a side note observation: “Now I know why Kevin is not on Twitter”. ;-)
Send me your thoughts.


  1. "Security" people in my experience have always been insular and self-secretive. "What are you working on, John?" "Can't tell you!"

    I would have recast the tweet above as: “Developers don't CARE shit about security”. Given my interactions with "security" people, my interest in their "security" concerns is nil.

    Dictates roll down from on-high, without much explanation, and just seem to "get in the way" of my work. No engagement, just mandates. "You MUST do it this way!" says "security". "Ok, ok, you win, just get the hell out of my way so I can get the real job done!". (Last part was my snark!)

  2. Superb post. On a death march at work preventing me from as comprehensive a response as it deserves but will get to it this weekend.

    While on the matter the names like Kevin Wall, Steve Taylor, Alex Russell, Ingo Struck and {fill in} were all at the forefront of OWASP in the early days (shame on me Kevin for not keeping in touch better). I would love to find the tipping point when you guys drifted away because you guys are exact.y the sort of people I think OWASP needs!!!

    A decent response later.

    (And yes I am embarrassed about the way I called out Rohits project but I am an ass sometimes. It's just me)

  3. Great write up.

    With regards to some of the projects that have been labeled by others as "failures" I too think some of the content within them is pretty good and just needs some re-factoring/re-purposing.

    Having been involved in a similar organization I think one of the more difficult aspects of open organizations is people to take roles in quality control/oversight. Finding experts to author good content isn't extremely difficult, finding leaders/people in oversight roles with 'enough' technical expertise is fairly difficult. I think this entire discussion (between the various blogs) is a good start in improving things, keep it going!

  4. I think so too this blog explains about a few things. You explain about the security for the software, the steps you take is really quite remarkable, you say "It's easier to teach a good developer of security than to teach someone how good security software development." I understand about the meaning of it, if that's the choice you want to go, you have a social life that is very high.