Response to the question: How to prevent double escaping in HTML

HTML double escaping is an issue encountered by developers who want to display user input, HTML entities, and when they create dynamic web pages. It occurs when the special characters are escaped more than once so that the browser displays the encoded text rather than the escaped characters.

For instance, the web page may present < instead of <. Upon re-escaped, may appear as &lt;. This can cause confused looks to content and confusion during troubleshooting of a website.

Having the knowledge of how the HTML escaping is achieved and the knowledge of the proper places to escape, to protect your application and avoid double escaping.

In this guide you’ll learn what double escaping is, why it occurs, how to avoid it and some guidelines to help keep HTML outputs correct.

What is Double Escaping in HTML?

Double escaping is escaping a character or string which is already escaped.

The special characters such as<, >, &, and quotation marks in HTML are used to tell markup or attributes. Often, to get these characters to appear as regular text, developers use HTML entities to represent them.

For example:

• < becomes <
• > becomes >
• & becomes &
• " can become "

The purpose of escaping them is to be able to output text that isn’t meant to be HTML. But the subsequent escape again and again may lead to an unexpected change in output!

Markdown is now going back to that.Markdown is now reverting to that.

Let’s say you want to show up the following text:

<hello>

It is then HTML escaped to become:

<hello>

If you manage to avoid the result again, it turns to:

<hello>

The browser leaves off the angle brackets and simply shows the entity sequences.

There isn’t much problem with escaping in itself. But the issue is, that the escape effort is being made at the wrong time or at times more often than needed.

Why Double Escaping Occurs?

Double escaping typically happens in apps where different components disagree that the string has been escaped.

These are some typical reasons.

2. Lost Data – More Than Once

A user might inadvertently enter harmful input, which could be escaped by a developer before it is stored and again when displayed.

An Application can convert the characters for example by having the Application transform “<” to “<” when saving a comment. Later that value will be automatically escaped before being rendered, thanks to a template.

This means that the user is able to view coded entities rather than the desired text.

How to avoid it: Use the same text during storing and processing non-sensitive content. Use the correct escaping for context of the value in an output.

2. Blending of the manual escaping with the automatic escaping.

A lot of template engines escape values that are inserted into the HTML for you.

Double escaping can result from manually escaping a string and then passing it to a template which also escapes it.

For example:

const message = "<hello>";

When a template is escaped by the automatic escape, these ampersands can be doubly escaped.

How to prevent it: Know how your template engine works in terms of escaping. Do not escape values to be escaped in the rendering system.

3. Storage of HTML Entities as data.

Here’s some of the issues if escaped HTML is saved in place of original data:

For instance if original text was <hello>, then the computed value of that database would be <hello>. The other element might render the stored value as regular text and escape it again.

How to avoid it: If possible, save unescaped plain text and rely on output layer to handle the context-appropriate escaping.

4. Incorrect Decoding and Re-encoding of Data

It is also possible for double escaping to occur, as one section escapes entities, and the other section escapes entities again.

The browser to render the HTML makes the task of forecasting the display very complex because of repeated encoding and decoding.

How to prevent it: Establish a clear process to receive, store, process and display data. Avoid needlessly encoding and decoding “for no reason”!

Double Escaping In HTML: How To Prevent

Preventing double escaping doesn’t imply that escaping is entirely prevented. The target is to get out of data at the time when it’s right, and not perform re-transformation for no reason.

Avoiding the loss of data in the Output Stage.

A great technique is to store normal app data without escaping it until the time of display.

For instance, put this original text:

const userInput = "<hello>";

Use safe DOM API when displaying as text in an html element:

const output = document.querySelector("output");

output.textContent = userInput;

HTML:

<div id="output"></div>

The browser will print out <hello> and not treat it as HTML.

You don’t have to manually add the code point for angle brackets to the textContent value. Takes the value provided as a textual value.

This will prevent unnecessary escaping and decrease HTML injection risks when presenting untrusted text.

However, it is important to use textContent and not innerHTML for Plain Text 2.

In JavaScript, there are a variety of methods by which an element’s content can be updated.

The difference between the two is the use of the textContent property to insert text and the use of the innerHTML property to insert a string of HTML content.

For example:

const message = "<strong>Hello</strong>";

document.querySelector("output").textContent = message;

The Browser shows the actual text:

<strong>Hello</strong>

Try modifier innerHTML now:

document.querySelector("output").innerHTML = message;

The string is interpreted as HTML, thus Hello gets rendered in bold by the browser.

If you want to display user typed text, textContent is more likely to be the best option. The benefit of using it is that there’s no need to HTML escape the string on assign.

Do not escape values that already have been escaped.

If you are given a string that’s already HTML-encoded as an entity:

const escapedText = "<hello>";

Note that ampersands could be replaced by & if you put the result of this string through another HTML escaping function.

It may turn out to be:

<hello>

To avoid this, check the first character of the value if it is an escaped value or the value itself is original text before processing.

Do not assume that any string with html entities requires more escaping nor that any string with html entities is safe.

A more effective strategy is to assign appropriate tasks to each component of the application and consider the proper output technique.

5. Understand Visual Basic Editor in Visual Studio

In many template languages, variables get escaped automatically when they are used to generate HTML.

For instance, a template could be written as:

<p>{{ userInput }}</p>

Some template engines automatically escape HTML sensitive characters from this. All of this depends on the particular configuration and system environment of the framework.

If you leave userInput out of your own control before you pass it to a template, it may be left out in the template again.

Make sure to avoid double escaping:

• Determine if your template engine automatically escapes the output.
• Do not escape the ordinary text, pass it into the template.
• Do not perform any unnecessary manual escapes.
• When displaying only trusted HTML is necessary, use framework supported methods to do so.

Avoid globally disabling automatic escaping to work around a display problem. This could introduce the possibility of cross-site scripting (XSS) vulnerabilities.

Use the Right Way of Escaping the Output Context

HTML escaping cannot be used everywhere data can go.

Each of these may need to be treated differently: Text in HTML elements, a quoted HTML attribute, a JavaScript string, a URL, and a CSS value.

So it is safe to alter a simple text value to an element like:

element.textContent = userInput;

If you have an HTML attribute, you may be able to skip string-based html construction and just set the property:

element.setAttribute("title", userInput);

Use the right kind of JavaScript serialization, not HTML escaping, to handle data in JavaScript. As with URLs, validate and limit the URL schemes and destinations, if suitable.

The solution would be to use a technique which needs to be applied based on the context at the destination, rather than indiscriminately escaping every value using HTML escaping.

6) Do not use repeated encoding/decoding functions.

Multiple layers of custom escaping functions may make applications more difficult to maintain.

One function might escape the HTML characters, a second might encode them, and a third might decode them before displaying them, for example. These functions may only be called in the correct order or the output is likely to be corrupted.

To prevent this:

• usar un mínimo de pasos de transformación.
• Pass one function an obvious reason.
• Do not escape anything prior to saving or on render if there is no reason to do so.
• View the final result in the browser.
• Instead of unnecessary custom escaping implementation using features of the established framework.

Single escaping becomes easier to avoid and detect with a consistent data-handling process.

Double Escaping in HTML Browser Extension By Widgets

Double escaping is the reason if your page is showing strings like &lt; to appear as <.

There are a few checks that can be made to determine the problem.

Make sure you are reviewing the Original Data.

Test the value as soon as it is received from a form, API or database.

When the original value is:

<hello>

Applications will receive, however,

<hello>

The value could have been escaped throughout the procedure.

2. Inspect the DOM

In the browser find the element in developer tools.

Include a check if it was inserted as text or HTML. Notice, however, that because of changes to the DOM, the Elements panel will appear as it does in the browser, whereas it may not be as it appears in the HTML source.

Check the Rendering Process 3.

Examine each step from getting the value to displaying the value.

Search for multiple invocations to escaping functions, encoding of already encoded values, and template-level automatic escaping.

4. Test Special Characters

Try simple sets of text strings with such characters as:

< > & " '

Carefully note the appearance of these after each processing step. This may assist you to detect an extra encoding process points.

Is it worth the trouble to Decode HTML Entities in order to fix double escaping?

Decoding HTML entities might appear to be a simple “solution,” but it isn’t necessarily.

One example is that substituting &lt; with < may solve one of the appearance issues. But repeated decoding of arbitrary data can lead to surprises and can cause potentially harmful markup to be revealed if the value is then output into an environment where HTML markup is attempted.

Do not attempt to retroactively undo each escape, but list all instances of unnecessary escaping in the source file, and address them in that line of the decoder.

Specify HTML decoding/rendering rules for content you are storing on purpose in HTML. Be mindful of untrusted HTML and if one must accept certain HTML blocks and elements use a trustworthy HTML sanitizer.

In typical UG content (such as textContent), it is usually easier and safer to rely on the naive escape semantics of an engine such as a template.

Typical errors that need to be avoided.

By avoiding some of the more common pitfalls, developers can eliminate many double-escaping issues.

• Escaping before storage, for an unclear reason: This can result in an additional escape of the data Nk later in the render layers.
• innerHTML for simple text: The string will be interpreted as HTML by browser, not as plain text.
• Turning off auto escaping: This may resolve a visual issue but compromise security.
• Decoding everything before display: this can receive data in code to be tagged for rendering by the browser.
• HTML escaping in all places: Contextualize the escaping for responding to JavaScript, CSS, URL and HTML.
• Assuming escaped data is always safe: Escaping is in context and encoded data can be unsafe in other contexts.

The best approach is to be aware of the way that data passes through your application and make sure every output layer performs appropriately with the data.

To prevent double escaping, please adhere to the following tips.

Here are some best practices to consider when designing websites and apps:

1. When processing and storing, maintain the text data as is and do not transform it.
2. Strip away escape values that would otherwise pose a problem if inserted, into an HTML output context, unless the selected API takes care of this problem.
3. Avoid using innerText if displaying plain text via JavaScript, use textContent instead.
4. Understand how template engine automatically escaping works.
5. Don’t escape any data that will be escaped for you.
6. To utilize various outputs such as attributes, urls, Javascript and others depending on context.
7. DON’T overrule security to fix formatting issues.
8. Write test strings with characters that are sensitive in HTML.
9. If your application purposefully allows some user supplied HTML use a trusted HTML sanitizer.
10. Articulate the whole rendering process when rendered entities occur out of the ordinary.

Conclusion

Double escaping in HTML is when special characters are encoded twice, which may be due to multiple components in the application trying to perform the same output transformation.

The safest approach to avoiding it is to store regular data as plain text, to be aware of what is automatically escaped and to use the correct output for each situation.

Generally, if you have to deal with plain text, textContent is an easier and safer choice in JavaScript. When using templates, always use all their standard escaping and don’t manually encode them unnecessarily.

These practices will ensure that your pages are correctly displayed, are easier to maintain, and will alleviate unwanted HTML entities from your page while protecting your application.