A common observation in front-end development is that newer developers often engage with modern programming paradigms without fully grasping their historical context.
It is understandable not to know every aspect of web development, as it is a vast field with diverse skills and specialties. Learning in this domain is a continuous process.
For instance, a developer might inquire about detecting when users leave a UI tab. The JavaScript beforeunload event is a solution, familiar to those who have encountered unsaved data alerts on other websites. The pageHide and visibilityChange events also offer relevant functionality.
Such knowledge often comes from practical project experience rather than initial JavaScript studies.
Modern front-end frameworks build upon preceding technologies, abstracting development practices for an improved developer experience. This often reduces or eliminates the need to deeply understand traditionally essential front-end concepts.
Take the CSS Object Model (CSSOM), for example. While one might expect extensive hands-on experience from CSS and JavaScript developers, this is not always true.
On a React e-commerce project, a stylesheet for a specific payment provider was loading on every page, though only needed on one. The developer assigned to fix this had no prior experience dynamically loading stylesheets, which is understandable given how React abstracts traditional methods.
While the CSSOM may not be an everyday necessity, interacting with it for specific, one-off tasks is probable.
These observations highlight numerous web features and technologies that developers might not encounter daily. Newer developers, especially, may be unaware of them due to the abstractions provided by modern frameworks.
The focus here is on XML, an older language with similarities to HTML.
This topic is relevant due to recent WHATWG discussions proposing the removal of a significant part of the XML stack, specifically XSLT, from browsers. Such older technologies could still address practical problems, like the CSSOM scenario previously mentioned.
Exploring XSLT, an older technology, might reveal how its features can be leveraged beyond XML for current real-world challenges.
XPath: The Central API
From an XML perspective, XPath stands out as a highly useful query language for locating any node or attribute within a markup tree with a single root element. While XSLT also relies on XPath, XPath itself holds greater importance for broader applications.
Arguments for XSLT’s removal typically omit XPath, suggesting its continued relevance. XPath is a crucial API within this technology suite, particularly for uses beyond standard XML. Its significance stems from its ability to locate elements that CSS selectors cannot, including those based on their current DOM position.
XPath offers this capability.
Some readers may be familiar with XPath, while others may not. XPath is an extensive technology, making it challenging to cover all its basics and advanced applications in a single article. An attempt to do so quickly exceeded typical article length limits.
This article will focus on practical XPath applications, providing resources for those interested in learning the fundamentals.
Combining XPath & CSS
XPath offers capabilities for querying elements that CSS selectors lack, though CSS selectors excel at querying by class name, a feature XPath doesn’t directly replicate.
CSSXPath.myClass/*[contains(@class, “myClass”)]
The CSS example targets elements with the .myClass class. In contrast, the XPath example queries elements where the class attribute contains the string “myClass”. This means XPath selects elements with .myClass and also those with “myClass” embedded in other class names, like .myClass2, making it a broader selection method.
This does not suggest replacing CSS with XPath for all element selections.
The key takeaway is that XPath offers unique capabilities not found in CSS and remains a valuable, albeit older, browser technology that might not be immediately apparent.
Combining these two technologies can provide new insights into XPath, adding a potentially overlooked tool to a developer’s skillset.
A challenge arises from the incompatibility between JavaScript’s document.evaluate method and the CSS API’s query selector methods.
A compatible querying API has been developed as a starting point, though it is a simplified example. Here is a basic reusable query constructor:
See the Pen queryXPath [forked] by Bryan Rasmussen.
Two methods, queryCSSSelectors (similar to querySelectorAll) and queryXPaths, have been added to the document object. Both return a queryResults object:
{
queryType: nodes | string | number | boolean,
results: any[] // html elements, xml elements, strings, numbers, booleans,
queryCSSSelectors: (query: string, amend: boolean) => queryResults,
queryXpaths: (query: string, amend: boolean) => queryResults
}
The queryCSSSelectors and queryXpaths functions execute the provided query on elements within the results array, assuming the array contains nodes. If not, an empty queryResult of type nodes is returned. When the amend property is true, the functions modify their own queryResults.
This implementation is for demonstration purposes only and should not be used in a production environment. Its sole purpose is to illustrate the combined effects of the two query APIs.
Example Queries
Several XPath query examples will be presented to showcase their powerful capabilities and potential as alternatives to other methods.
Consider the XPath query //li/text(). This expression targets all <li> elements and extracts their text nodes. For instance, given the following HTML:
<ul>
<li>one</li>
<li>two</li>
<li>three</li>
</ul>
The result would be:
{"queryType":"xpathEvaluate","results":["one","two","three"],"resultType":"string"}
This translates to the array: ["one", "two", "three"].
Typically, one would query for <li> elements, convert the result to an array, then map over it to extract each element’s text node. XPath offers a more concise way to achieve this:
document.queryXPaths("//li/text()").results.
The text() function, resembling a function signature, is used to retrieve an element’s text node. In the example, it extracts the text (‘one’, ‘two’, ‘three’) from each of the three <li> elements.
Another text() query example, using the following markup:
<pa href="/login.html">Sign In</a>
To retrieve the href attribute value, the following query can be used:
document.queryXPaths("//a[text() = 'Sign In']/@href").results.
Similar to the previous example, this XPath query on the current document returns the href attribute of an <a> element containing the text “Sign In”. The result is ["/login.html"].
XPath Functions Overview
XPath includes various functions, some of which may be unfamiliar but are particularly useful:
- starts-with If a text starts with a particular other text example,
starts-with(@href, 'http:')returns true if an href attribute starts with http:. - contains If a text contains a particular other text example,
contains(text(), "Smashing Magazine")returns true if a text node contains the words “Smashing Magazine” in it anywhere. - count Returns a count of how many matches there are to a query. For example,
count(//*[starts-with(@href, 'http:'])returns a count of how many links in the context node have elements with an href attribute that contains the text beginning with the http:. - substring Works like JavaScript substring, except you pass the string as an argument. For example,
substring("my text", 2, 4)returns “y t”. - substring-before Returns the part of a string before another string. For example,
substing-before("my text", " ")returns “my”. Similarly,substring-before("hi","bye")returns an empty string. - substring-after Returns the part of a string after another string. For example,
substing-after("my text", " ")returns “text”. Similarly,substring-after("hi","bye")returns an empty string. - normalize-space Returns the argument string with whitespace normalized by stripping leading and trailing whitespace and replacing sequences of whitespace characters by a single space.
- not Returns a boolean true if the argument is false, otherwise false.
- true Returns boolean true.
- false Returns boolean false.
- concat The same thing as JavaScript concat, except you do not run it as a method on a string. Instead, you put in all the strings you want to concatenate.
- string-length This is not the same as JavaScript string-length, but rather returns the length of the string it is given as an argument.
- translate This takes a string and changes the second argument to the third argument. For example,
translate("abcdef", "abc", "XYZ")outputs XYZdef.
Beyond these specific XPath functions, many others operate similarly to their JavaScript or general programming language equivalents, such as floor, ceiling, round, and sum, which can also be quite useful.
The following demo illustrates each of these functions:
See the Pen XPath Numerical functions [forked] by Bryan Rasmussen.
Similar to most string manipulation functions, many numerical XPath functions accept a single input. This design is intended for querying, as demonstrated in the last XPath example:
//li[floor(text()) > 250]/@val
When used as shown in most examples, these functions will operate on the first node that matches the specified path.
While type conversion functions exist, they should generally be approached with caution due to JavaScript’s existing type conversion complexities. However, there are scenarios where converting a string to a number for comparison might be necessary.
Key XPath datatypes include boolean, number, string, and node, with corresponding functions to set these types.
Many XPath functions can operate on non-DOM node datatypes. For instance, substring-after accepts a string, which could be from an href attribute or a standalone string:
const testSubstringAfter = document.queryXPaths("substring-after('hello world',' ')");
This example will yield the result array ["world"]. A demo page illustrates functions used against non-DOM nodes:
See the Pen queryXPath [forked] by Bryan Rasmussen.
A notable behavior of the translate function is that any character present in the second argument (the characters to be translated) that lacks a corresponding translation in the third argument will be removed from the output.
For example, this expression:
translate('Hello, My Name is Inigo Montoya, you killed my father, prepare to die','abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ,','*')
…produces the following string, including spaces:
[" * * ** "]
Here, the letter ‘a’ is translated to an asterisk (*), while all other characters without a specified translation are removed entirely. Only the whitespace remains between the translated ‘a’ characters.
Conversely, this query:
translate('Hello, My Name is Inigo Montoya, you killed my father, prepare to die','abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ,','**************************************************')")
…avoids this issue, yielding a result like this:
"***** ** **** ** ***** ******* *** ****** ** ****** ******* ** ***"
It may seem that JavaScript lacks a direct equivalent to XPath’s translate function, though replaceAll with regular expressions can address many similar use cases.
While the demonstrated approach is possible, it is suboptimal for simple string translation. The following demo provides a JavaScript wrapper for XPath’s translate function:
See the Pen translate function [forked] by Bryan Rasmussen.
One application for this could be Caesar Cipher encryption with a three-place offset, a historical encryption method:
translate("Caesar is planning to cross the Rubicon!",
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz",
"XYZABCDEFGHIJKLMNOPQRSTUVWxyzabcdefghijklmnopqrstuvw")
Inputting “Caesar is planning to cross the Rubicon!” yields “Zxbpxo fp mixkkfkd ql zolpp qeb Oryfzlk!”
As another example, a ‘metal’ function was created to take a string input and use translate to return the text with all characters that can take umlauts.
See the Pen metal function [forked] by Bryan Rasmussen.
const metal = (str) => {
return translate(str, "AOUaou","ÄÖÜäöü");
}
When given “Motley Crue rules, rock on dudes!”, it returns “Mötley Crüe rüles, röck ön düdes!”
This function could inspire various parody uses, and the TVTropes article offers ample inspiration.
Using CSS With XPath
The primary motivation for combining CSS selectors with XPath is that CSS inherently understands classes, while XPath’s class handling relies on string comparisons of the class attribute, which is usually sufficient.
However, issues could arise if classes like .primaryLinks and .primaryLinks2 exist, and XPath is used to target .primaryLinks. While XPath is generally suitable in the absence of such naming conventions, such scenarios are not uncommon in real-world development.
Another demo illustrates the combined use of CSS and XPath, specifically when running an XPath query on a context node other than the document’s root.
See the Pen css and xpath together [forked] by Bryan Rasmussen.
The CSS query .relatedarticles a retrieves two <a> elements within a <div> that has the .relatedarticles class.
Following this are three “problematic” queries that do not yield the desired results when executed with these elements as the context node.
The unexpected behavior of these three queries can be explained:
//text(): Returns all the text in the document.//a/text(): Returns all the text inside of links in the document../a/text(): Returns no results.
These results occur because, despite the context being <a> elements from the CSS query, // searches the entire document. This highlights XPath’s power: unlike CSS, it can traverse from a node up to an ancestor, then to a sibling of that ancestor, and finally down to a descendant.
Conversely, ./ queries the children of the current node, where . signifies the current node and / indicates a child node (attribute, element, or text, depending on the path). Since no child <a> element is selected by the CSS query, this query yields no results.
The last demo includes three effective queries:
.//text(),./text(),normalize-space(./text()).
The normalize-space query not only illustrates XPath function usage but also resolves an issue present in the other queries. The HTML structure is as follows:
<a href="https://www.smashingmagazine.com/2018/04/feature-testing-selenium-webdriver/">
Automating Your Feature Testing With Selenium WebDriver
</a>
The original query returns text nodes with leading and trailing line feeds, which normalize-space effectively removes.
Applying an XPath function that returns a non-boolean value to an input XPath is a common pattern. The following demo provides several examples:
See the Pen xpath functions examples [forked] by Bryan Rasmussen.
The first example highlights a potential issue to be aware of. Specifically, the following code snippet:
document.queryXPaths("substring-after(//a/@href,'https://')");
…yields a single string:
"www.smashingmagazine.com/2018/04/feature-testing-selenium-webdriver/"
This behavior is logical: these functions return single strings or numbers, not arrays. When applied to multiple results, only the first result is returned.
The second result demonstrates the desired outcome:
document.queryCSSSelectors("a").queryXPaths("substring-after(./@href,'https://')");
This returns an array containing two strings:
["www.smashingmagazine.com/2018/04/feature-testing-selenium-webdriver/","www.smashingmagazine.com/2022/11/automated-test-results-improve-accessibility/"]
XPath functions support nesting, similar to JavaScript. Knowing the Smashing Magazine URL structure, one could construct the following query (template literals are recommended):
`translate(
substring(
substring-after(./@href, ‘www.smashingmagazine.com/')
,9),
'/','')`
This complex query extracts the URL path from the href attribute after www.smashingmagazine.com/, removes the initial nine characters, and then translates forward slashes (/) to an empty string to eliminate trailing slashes.
The resulting array is:
["feature-testing-selenium-webdriver","automated-test-results-improve-accessibility"]
More XPath Use Cases
XPath proves exceptionally valuable in testing. Its strength lies in its ability to access any element in the DOM from any position, a capability not fully matched by CSS.
In modern build systems, CSS classes may not always be consistent. XPath, however, allows for more robust matching of an element’s text content, independent of DOM structure changes.
Research has explored techniques for creating resilient XPath tests. Test failures due to changed or removed CSS selectors are a common frustration.
XPath also excels at multiple locator extraction. While both XPath and CSS offer multiple ways to match an element, XPath queries can be more targeted, refining results to find specific matches even when several possibilities exist.
For instance, XPath can retrieve a specific <h2> element nested within a <div> that directly follows a sibling <div>, which itself contains a child <img> element with a data-testID='leader' attribute:
<div>
<div>
<h1>don't get this headline</h1>
</div>
<div>
<h2>Don't get this headline either</h2>
</div>
<div>
<h2>The header for the leader image</h2>
</div>
<div>
<img data-testID="leader" src="image.jpg"/>
</div>
</div>
The corresponding query is:
document.queryXPaths(`
//div[
following-sibling::div[1]
/img[@data-testID='leader']
]
/h2/
text()
`);
A demo illustrates this concept:
See the Pen Complex H2 Query [forked] by Bryan Rasmussen.
Indeed, XPath provides numerous ways to target any element within a test.
XSLT 1.0 Deprecation
Earlier, it was noted that the Chrome team intends to remove XSLT 1.0 support from browsers. This is significant because XSLT 1.0, found in most browsers, relies on XPath 1.0 for XML-focused document transformation.
This change would remove a key XPath component. However, given XPath’s utility in writing tests, its complete disappearance seems improbable.
Interestingly, features often gain attention upon deprecation, as seen with XSLT 1.0. A Hacker News discussion features numerous arguments against its removal, with one post demonstrating XSLT’s use in a blogging framework and exploring JavaScript shims for such scenarios.
Suggestions have also emerged for browsers to adopt SaxonJS, a JavaScript port of Saxon’s XSLT, XQuery, and XPath engines. This is compelling because SaxonJS implements current versions of these specifications, unlike browsers which typically support only XPath and XSLT 1.0 and no XQuery.
Norm Tovey-Walsh of Saxonica, creators of SaxonJS and other Saxon engine versions, commented:
“If any browser vendor was interested in taking SaxonJS as a starting point for integrating modern XML technologies into the browser, we’d be thrilled to discuss it with them.”
He further stated:
“I would be very surprised if anyone thought that taking SaxonJS in its current form and dropping it into the browser build unchanged would be the ideal approach. A browser vendor, by nature of the fact that they build the browser, could approach the integration at a much deeper level than we can ‘from the outside’.”
These comments from Tovey-Walsh were made approximately a week prior to the XSLT deprecation announcement.
Conclusion
This discussion aims to highlight the power of XPath and provide numerous examples of its effective use. It serves as an excellent illustration of how older browser technologies can retain significant utility today, even if they are unfamiliar or not typically considered.
Further Reading
- “Enhancing the Resiliency of Automated Web Tests with Natural Language” (ACM Digital Library) by Maroun Ayli, Youssef Bakouny, Nader Jalloul, and Rima Kilany This article provides many XPath examples for writing resilient tests.
- XPath (MDN) This is an excellent place to start if you want a technical explanation detailing how XPath works.
- XPath Tutorial (ZVON) This tutorial has been found to be highly helpful for learning, thanks to its wealth of examples and clear explanations.
- XPather This interactive tool lets you work directly with the code.
