Update March 1st: Since the PDF on RSA website seems broken, I have attached a version to this blog post.



I'll be at RSA this week. My session is Friday morning (10:20am, Room 132) and is called:

Why haven't we stamped out XSS and SQL yet?


RSA talk content

Since all the slides are apparently available for everyone on the RSA website, I can give some more insight about what I will be talking about. We ran an experiment at Coverity in which we analyzed many Java web applications and looked for where developers add dynamic data. The goal was to try to understand what contexts (both HTML contexts and SQL contexts) are frequently used.


The tone of the talk is fairly straightforward: security pros. have been giving advice to developers for a long time, yet we still have these issues on a frequent basis, so we map common advice with what we see from the data.


What you can expect from this talk:

  • Some information about observed HTML contexts: that's about 26 different contexts stacks, 45% of them had 2 elements in the stack (e.g., HTML attribute -> CSS code), and the longest ones had 3 elements.
  • A list of SQL contexts and good notes about what developers usually do.
  • Advice for security pros. on how to communicate with developers (things that led to the creation of our Fixing XSS: A practical guide for developers document).


Anyhow, this blog post is not only to announce this talk, but also to give some insight on how we extracted the data from these applications.


Analysis technique

We created and modified different checkers from Coverity Security Advisor in order to extract all injection sites that are related to dynamic data regardless of its taintedness. For each injection site, we computed the context in which it belonged to the sub language (one of HTML, JavaScript, CSS, SQL, HQL, and JQPL). This represents our working dataset.


Here's an example of an injection site (using JSP and EL):

<script type="text/javascript">
var content = '${dynamic_data}';
// context ::= {HTML SCRIPT TAG -> JS STRING}


We tracked the construction of this snippet of the HTML page and recorded the injection site such as ${dynamic_data} and its associated context (JS STRING inside HTML SCRIPT TAG). Since we do not care about the taintedness of dynamic_data, we didn't need to track all paths that could lead to a defect (XSS here), and that's where what we did is very different from our XSS checker.

Note that we still needed to properly track parts of the HTML page that's being constructed to properly compute the context. This is however part of our context-aware global data flow analysis...


For SQL related queries, we essentially needed to do the same thing, but we also needed to track the parameters being inserted in a query using a parameterized notation: remember, we needed to find all dynamic data that could eventually go into a query.

That's why the following code:

String sql = "select foo, bar from table where 1=1";
if (cond1)
  sql += " and user='" + user_name + "'"; // context ::= {SQL_STRING}
if (cond2)
  sql += " and password=?"; // context ::= {SQL_DATA_VALUE}

has 2 interesting injection sites for the experiment, and we didn't need to understand the full abstract string (an eventual set of 4 possible strings) from this piece of code.


Note that if there is this fairly common construct:

String sql1 = "select foo, bar from table where ";
String and_beg = " and (";
String and_end = " ) ";
sql1 += and_beg + "user = '" + user_name + "'" + and_end;
sql1 += sql2; // `sql2` is another part of the query coming
              // from a different procedure or so

we will still properly track the contexts even if all parts (sql1, and_beg, etc.) are inter-procedurally created.



I will quickly explain this during the talk, but essentially tracking HTML contexts on a global data flow analysis is not a trivial thing. Moreover, considering the impact of some JavaScript code on the resulting web page (and therefore where the HTML contexts could potentially be transformed at runtime) is an even more complex problem. We did not analyze JavaScript.

I'm happy to announce a new document we just made available: Fixing XSS: a practical guide for developers. If you're currently at the RSA conference, you should come to Coverity's booth (#1759) and either get a hardcopy or a USB stick with this document on it.


As the title suggests, this document is a guide for developers on how to handle dynamic data in various locations and common constructs in HTML. We leveraged the data we got from our research for our talk at RSA to come up with some of the most common HTML contexts and nested contexts, and improved the Coverity Security Library to have a solution for all of these cases.


Looking at the documentation available for XSS, several things strike us:

  1. It often talks about how to exploit an XSS and not how to fix this issue.
  2. The HTML contexts information is always lacking precision and often makes the documentation complex to read (we're also guilty of this in some previous blog posts).
  3. The fixes are limited or too restrictive (i.e., not applicable for developers).

That's mostly why we decided to create our own document for developers.


The first release of this document contains 13 common HTML constructs, and we plan on adding more to it. We also describe what HTML contexts are and why it's important to think about them when outputting dynamic data in a web page. However, we also want to create collateral that gives more complete information about HTML contexts and why it matters for XSS.


In this document, you can expect to learn what happens when you want to add dynamic data in a HTML context such as HTML snippet inside a JavaScript string:

<div id="forMyContent"></div>
  var foo = "<h1>${cov:jsStringEscape(cov:htmlEscape(content))}</h1>";

and why you need to first use an HTML escaper, then a JavaScript string escaper.


You'll also see the usage of a newly introduced function asUrl from CSL that helps writing fully dynamic URLs inside an HTML attribute such as:

<a href="${cov:htmlEscape(cov:asURL(content))}">
  Click me


The current document uses the Java Expression Language (EL) notation to show the dynamic data (here ${content}), but all functions are also available directly from Java when using CSL.


Whether you develop web applications, have developers, or do security review, you should read and share this document. We're also happy to receive any feedback to keep improving this document.

Alright, this is a bit of a different post. Summer approaches, and we are looking for an intern.


If you're interested in security and program analysis, and have a good background in one of the two fields, then please reach out to us (I believe you can send DM on this blog, but otherwise, you can contact me at rgaucher@coverity.com).


We have several ideas that could be developed based on the skills of the individual:

  • Pushing the limits of our dynamic whitebox sanitization fuzzer
  • Improvement of generation and placement of remediation advices
  • Identifying string constraints on tainted input for some set of sinks
  • Etc.


Also, if you have a cool idea (related to program analysis & security obviously) that you'd want to work on for a couple of months, we're happy to hear it.

Over the past year or so, the HTML 5 specification has been a non-friendly but necessary reference to me (/us). Indeed, this is the only place that really explains how an HTML 5 document gets tokenized (a necessary step before parsing).


If you're doing research related to XSS or HTML contexts, and you never had a look at this document, I suggest you go ahead and dive into it. That's mostly the key to finding something like the script data double escaped state as described by Jon.


However, if you had a serious look into it, I'm sure you had one of these reactions:

Where am I know?

How did I end up here?

So, just for the sake of making our life easier, I created a small visualization page for the spec. I mostly scraped the tokenizer spec, and generated a graph for it.


The result is a self-contained HTML document that helps you navigate through the tokenization specification, and lets you click on states and remembers where you're coming from. It's really just to make our life easier:



This document is available here: HTML5 tokenization visualization.


If you're interested in how to get the data, we published the script that generates a JSON or DOT file on Github: security/html5-tokenizer-extraction at master · coverity/security · GitHub

Getting the right HTML context is not always easy to say the least. It’s often easy to forget to apply some escaping, such as for one of the parent HTML contexts. That’s where Coverity Security Advisor comes into play. Our static analysis for cross-site scripting is HTML context aware, and will provide very accurate remediation advices on what escaper(s) to use or sanitization to leverage.


I want to emphasize that there is no dependency between the Coverity Security Library (CSL) escapers we provide and the Coverity products. We developed these escapers because there is no simple and good solution available for most java web developers. Also, we found it easier to mention these escapers in the remediation advices from Security Advisor when appropriate.


To illustrate the relation between CSL and Security Advisor, here is an example of what you can get out of it, and how the remediation advice really helps fixing the defects. The application is very simple and uses Spring MVC 3, JPA, JSP and EL; a very common web application stack.


In this case, I told the analysis not to trust the database, which is not our default trust model.


In the application, I first have a Spring MVC entry point that makes a query based on the title of a blog entry and returns a list of candidate (approximate search based on supplied title). I select the first one (line 53), extract the title, and add it to the model (line 56) under the name found_title. If you’re not familiar with Spring MVC, the model variable will essentially be added to the request attribute, and be available from expression language (EL).




Then the entry point dispatches to a JSP file (entry.jsp):


The ${found_title} EL variable is the title that we retrieved from the database (and it is considered tainted as this is what I instructed the analysis to do).  Some tests are done to see if we should display or not a banner. We take the true branch in the test and consider the body of the <c:if> tag (line 7) to be displayed.


This is where we have a stored cross-site scripting defect that requires a non-trivial fix due to nested HTML contexts. In this case, the tainted value ${found_title} is appended in a jQuery selector (approximated to a JavaScript string by our current HTML context parser), which is inside the onclick DOM event.


The analysis reports the stack of HTML contexts that are associated with this tainted data:

  1. HTML_ATTR_VAL_DQ: HTML double quoted attribute value
  2. JS_STRING_SQ: JavaScript single quoted string


Also, the analysis reports the appropriate remediation pointing in this case to the Coverity Security Library and the JSTL: the fix is to actually use the EL construct ${fn:escapeXml(cov:jsStringEscape(found_title))} instead of using directly the ${found_title} variable.


The recommendation is to:

  1. Use an escaper for the JavaScript string so that the value of found_title cannot escape out of the JavaScript string: the EL Coverity escaper cov:jsStringEscape
  2. Use an HTML escaper to ensure that the HTML attribute (outer context) cannot be escaped out: the JSTL HTML escaper fn:escapeXml


Et voila! That's what I call enabling good code.


Note: Writing this post, I realized that in the events we report "Cross-site script (XSS) injection (XSS)" which is quite incorrect; this is has been corrected since.

Escapers are part of the toolset for any developer. They write in a language that has rules, and when writing in a string, escape sequences are required for some characters to appear in it. The CS-101 example (at the time they taught C as the first language) is the backslash escaper for C strings: if you want a double quote to appear in a C string, you need to write "\"".


Language grammars usually define what is acceptable as escape characters and web languages such as HTML, CSS, and JavaScript do not deviate from this rule: they all have string escapers. For example, ECMAScript defines several escape sequences for JavaScript string, some of which are: backslash escape \', hex escape \x27, and Unicode escape \u0027.


Most of the cross-site scripting (XSS) defects can be fixed by using the right escaper(s) and in the right order. However, to know which to use we need to understand what HTML contexts are involved.


Let’s take the example of writing a JavaScript string inside a <script> tag from a JSP page (using EL). First, we need a JavaScript string escaper to ensure the string cannot be terminated by a user input, but still make sure that the parent context cannot be escaped out by having the string literal </script> inside this string.


Practically, it means that instead of writing the obviously wrong:

<script type="text/javascript">
logBlogComment($("#comment").text(), "${tainted_var}");


or the incorrect:

<script type="text/javascript">
logBlogComment($("#comment").text(), "${fn:escapeXml(tainted_var)}");


we should use the JavaScript string escaper:

<script type="text/javascript">
logBlogComment($("#comment").text(), "${cov:jsStringEscape(tainted_var)}");


To help address such issue, we developed the Coverity Security Library (CSL), a set of escapers that are easily integrated in any Java application and made them freely available on github. In addition we used a BSD-style license so that everyone can just take it and/or modify with no real constraint.

These escapers are lightweight and fast, do not require any configuration, and are well tested. Expression Language (EL) bindings are available in addition to the Java classes to make it easier to use in JSP.


Feel free to ask for new features, or send your pull requests.


Important links:

If there is something that security persons are not good at is to give good remediation advices. From my personal experience, I wasn’t able give proper remediation advices until I became intimate with the application and/or the frameworks it uses. So yes – disclaimer – I was one of these security consultants who had this kind of chat with developers:

Developer: Okay, so what can I do to fix this cross-site scripting here?

Me: You need to validate your input and make sure to encode the output properly; that is using HTML encoding.



I was wrong in many ways when communicating that way with developers:

  1. I did not use the right vocabulary, so I got some confused (I need to use “escaping” instead of “encoding” as “encoding” has a very different meaning);
  2. I gave a very generic advice about input validation and “encoding” the output, which did not translate properly in their mind since they did not know what I was talking about;
  3. I was also mostly wrong about input validation, it’s only important in few cases that are not the most common for XSS;
  4. I did not try to ask more questions about their application, what framework they use, and how it would be possible for them to fix the issue with few examples.


I believe it took me some time to get on the right track, but as I became more and more intimate with XSS as well as with web app development (mostly PHP or Python though), it started to all make sense (to the extent that it can make sense to anyone).


As a developer, I do not care for generic advice. Actually, when I’m looking for code snippets or searching stackoverflow for some issue I have with a language, framework, etc., I search for precise answers. Generic advice doesn’t play well if I want to be efficient. It’s okay though if I’m willing to spend the time thinking about the problem and come up with my own solution to it.


I believe however this is exactly what we should not inspire developers to do: coming up with their own solution to the cross-site scripting (or any injection type of issue) problem. Cross-site scripting is actually easy to argue for: think about the number of HTML contexts there are (we enumerated 30+, but about 5 are very common for dynamic data), how many times these can be nested in a different language (3 common languages in a web page), and you get the size of the problem you need to solve.


To understand better how wrong I was, let’s take the example of a cross-site scripting occurring in a style attribute. Here’s a typical part of a Java web application (using EL in JSP) that generates such issue:


<div style="background:url('/img/${tainted_var}.png')">
  Some content here.


For non Java-aficionados, ${tainted_var} is an EL construct that is dynamically evaluated to fetch the content of the variable tainted_var stored in some attribute map (usually the request or page context attributes). The content of the tainted_var attribute is directly concatenated to the HTML, so inside the style attribute.


A typical exploit for such issue would be to close the CSS string, and insert a new CSS directive that could evaluate JavaScript. Another common technique is to close the HTML style attribute an insert a new one such as a DOM event that will execute JavaScript:

  • Escape the CSS string and inject an expression directive: '); x:expression(alert(1));d:url('X
  • Escape out of the style attribute: " onclick="alert(1)" x="

Both payloads will result in the same execution of JavaScript: pops up an alert box containing '1' (yeah, I know... fancy).


Applying the remediation I gave to the developer, I would mostly recommend HTML escaping in that case, and the new code would become something like this:


<div style="background:url('img/${fn:escapeXml(tainted_var)}.png')">
  Some content here.


The fn:escapeXml function is provided by the JSTL and HTML escapes the following character into their HTML entities: ' " < > &


In that case, even if the second payload which tries to escape out of the HTML style attribute would not work anymore since the content of the style attribute is properly escaped for the HTML attribute, the first payload targeting the CSS syntax still works since the web browser interprets the HTML entities before passing the data to the CSS engine. My remediation did not work, and I did not help the developer fixing his problem; instead I made it arguably harder to test for: in short, I failed them.


Of course the problem can be addressed by doing a strong input validation, such as allowing only digits and letters, but I then add restrictions on the data. If the restrictions merely enforce assumptions made in the requirements for the application, that’s great and it might help prevent some vulnerabilities. But if these restrictions are in conflict with the requirements of the application, it might not be possible to restrict to a safe subset. In short, input validation is a good practice that can contribute to security, but it’s not (usually) where the buck stops. The unfortunate reality is that developers sometimes get confused that "input validation" is enough by itself, even when it’s not.


Fortunately in this case, there is a way to fix the issue regardless of input validation: apply the correct escapers in the right order. For the payloads not to work, we need to first make sure that the CSS string cannot be escaped out, and then that the HTML style attribute cannot be closed. For this, we need to use two escapers:

  1. CSS string escaper
  2. HTML escaper


The right solution becomes something like this:


<div style="background:url('/img/${fn:escapeXml(cov:cssStringEscape(tainted_var))}.png')">
  Some content here.


Where the cov:cssStringEscape performs CSS string escaping on the value of tainted_var to ensure that the CSS string cannot be closed by a value of tainted_var. The output of this escaper is then passed to the HTML escaper that ensures that the computed string cannot close the style attribute.


Cross-site scripting is an instance of a general problem of string concatenation enabling tainted data to masquerade as part of the output language it’s embedded in. Escaping ensures the data is treated as data, not as the output language.


Our new security product, Security Advisor, has been designed from the beginning with this thought and eventually, all the technologies we created for this new product have one main goal: precisely point to the issue and tell developers how to write their code properly. The level of precision we currently have for cross-site scripting or SQL injection allows us to give actionable remediation advices:

  • Without our whitebox fuzzer, we could not automatically determine the escapers available in the code base, or if the candidate defect (from the analysis perspective) is actually fixed due to the right escaper(s) being used.
  • Without the HTML context parser tied to our dataflow analysis, we would not be able to exactly understand where the data is going to in an HTML page and would not be able to see what escaper is required.


Finally, we created a remediation engine, which essentially gets a lot of different information from the analysis, and determines (or constructs) the appropriate remediation for the particular defect. It’s really several cool pieces of technologies that we put together and allows us to provide actionable remediation to the developers.


This blog post is the beginning of a series around Security Advisor, and what it's made of. Especially, I will talk about the HTML context parser, what the whitebox fuzzer really does, and examples of accurate remediation advices.

A URL spoofing bug affecting iOS 5.1 was recently found by Major Security (link to original PoC). Several news already covered the subject, but the following question on Hacker Newswas interesting:

Since you're opening a new window from another domain and writing arbitrary HTML into it, I wonder if this vulnerability could be used to bypass cross-domain restrictions...

Quick summary of the bug

This issue is a typical URL spoofing bug where the displayed URL does not reflect the actual domain where the page lives. For example, this bug allows me to do this:

To get this page, you need to spoof the URL in a window.open call. Here's the snippet of code that will generate this previous page:

  w = window.open('https://www.google.com', 'Google');
  w.document.write('<html><style>body{background-color:yellow;font-family: sherif;color:blue;font-size:100px}</style>Welcome to the new google.com</html>');

Can this bug be used to interact with the spoofed domain?

The question on Hacker News is actually legitimate: since the domain we see in the URL address bar is www.google.com, can the HTML/JavaScript code actually interact with the actual www.google.com pages?

Fortunately, the answer is no. If this is the case, the bug is much more serious, and would be a bypass of the Same Origin Policy (SOP). Simply put, SOP is here to prevent pages from a domain to access another domain's DOM. That means that if you go to www.example.com, this site should not be able to access your current DOM (let's say, your cookies) from www.twitter.com.

So, how to verify that fact? Well, a quick test is to look at what domain this code is actually hosted. If the domain (document.domain) is different than the URL's domain, this is where a spoofing appear. If the DOM's domain is changed, we fall in the more serious case described earlier (SOP bypass). We can display the current domain as understood by the same origin policy by using the following snippet as body of the spoofed page: alert('domain: ' + document.domain)

And here we go:

Impact of the bug

This kind of bugs was quite popular several years ago, from null-byte injection in the URL bar to using Unicode. This one is does not seem particularly interesting in term of cause: the window handler is left for writing (for same domain popups) as usual, but if in this case, the URL might not properly refreshed.
The real problem however, is that this bug facilitates any phishing attack. Think about it: it looks like you go to www.google.com (yeah, that's what the URL bar says!), but you're eventually on a site that could proxy each request to www.google.com: good-bye the secrecy of your passwords, emails, etc.


Until this bug gets fixed in a new iOS update by Apple, make sure to block the popup in the Safari options; that should be good enough for the time being, and make sure that you have the HTTPS visual indicator.