Why is //customer[@id=$x] inside a loop slow, and how do you fix it in XSLT?

Each evaluation scans the whole document, making the loop quadratic. Declare an xsl:key on customer/@id and use key('name', $x), which builds an index once.

The Report That Took 40 Minutes — XML in Production

Scenario

Example team: a finance script joins orders to customers in one XML export: for each <order customer="C17"> it finds <customer id="C17">. Tests use 50 orders and 20 customers.

What you see in production

The month-end run with 60,000 orders and 25,000 customers takes 40 minutes, pegging a CPU core. The finance team starts it the night before "just in case".

The naive code

for order in doc.iter("order"):
    customer = doc.xpath(f"//customer[@id='{order.get('customer')}']")[0]   # scans the whole document

Reproduce it

Counting the elements each approach touches shows the shape of the problem:

Nested scans vs an indexTry it yourself

from lxml import etree

def make(orders, customers):
    cs = "".join(f'<customer id="C{i}"><name>Customer {i}</name></customer>' for i in range(customers))
    os_ = "".join(f'<order id="O{i}" customer="C{i % customers}" total="{i % 90 + 10}"/>' for i in range(orders))
    return etree.fromstring(f"<export><customers>{cs}</customers><orders>{os_}</orders></export>".encode())

for orders, customers in [(50, 20), (2000, 800)]:
    doc = make(orders, customers)
    size = sum(1 for _ in doc.iter())
    naive_work = orders * size                          # each //customer[...] walks the whole tree
    index = {c.get("id"): c.findtext("name") for c in doc.iter("customer")}   # one pass
    indexed_work = size + orders
    joined = sum(1 for o in doc.iter("order") if o.get("customer") in index)
    print(f"{orders} orders: naive ~{naive_work:,} element visits, indexed ~{indexed_work:,} ({joined} joined)")

Output

50 orders: naive ~4,650 element visits, indexed ~143 (50 joined)
2000 orders: naive ~7,206,000 element visits, indexed ~5,603 (2000 joined)

Root cause

//customer[@id=…] searches the whole document on every call, so the loop is O(orders x document size): quadratic. XPath 1.0 engines in libxml2, the JDK and browsers don't build indexes for you. The same trap exists in XSLT with //customer[@id = current()/@customer] inside xsl:for-each.

The fix

Build a dictionary (or hash map) of customers in one pass, then look up each order: O(n).
In XSLT, declare <xsl:key name="customer" match="customer" use="@id"/> and use key('customer', @customer).
Use relative paths (./name, not //name) inside loops.
If the data is really relational, load it into a database and join there.

Trade-offs

An index costs memory proportional to the customers, which is usually fine.
Keys in XSLT are built per document and per key; they pay off when looked up more than a few times.

Detect it in production

Track job duration against input size; superlinear growth is the signature.
Profile with real data volumes before month-end, not sample files.

Key points

// inside a loop is a full-document scan every iteration.
Index once (dict, xsl:key), then look up.
Test performance with production-sized data.