The Import That Found Zero Orders

Scenario

Example team: an e-commerce company imports supplier orders every night from an XML file the supplier uploads. The import code was written against a sample file and has run without trouble for two years. The supplier upgrades its ERP system, and the new export starts with <orders xmlns="urn:supplier:orders:v2">.

What you see in production

Nothing fails. The import job finishes in seconds, logs "imported 0 orders" and exits with status 0. Two days later the warehouse asks why no supplier stock has arrived; customers are already seeing "out of stock".

The naive code

for order in doc.xpath("//order"):          # no namespace
    save(order.findtext("id"), order.findtext("total"))
log.info("imported %d orders", count)        # 0 is treated as a normal result

Reproduce it

from lxml import etree

old = b"<orders><order><id>A1</id><total>40.00</total></order><order><id>A2</id><total>12.50</total></order></orders>"
new = b'<orders xmlns="urn:supplier:orders:v2"><order><id>A1</id><total>40.00</total></order><order><id>A2</id><total>12.50</total></order></orders>'

for label, data in (("old feed", old), ("new feed", new)):
    doc = etree.fromstring(data)
    print(f"{label}: root={doc.tag!r} naive //order -> {len(doc.xpath('//order'))} orders")

# The fix: query the namespace explicitly (and fail loudly on an empty import)
NS = {"o": "urn:supplier:orders:v2"}
doc = etree.fromstring(new)
orders = doc.xpath("//o:order", namespaces=NS)
print("namespace-aware:", [(o.findtext("o:id", namespaces=NS), o.findtext("o:total", namespaces=NS)) for o in orders])

old feed: root='orders' naive //order -> 2 orders
new feed: root='{urn:supplier:orders:v2}orders' naive //order -> 0 orders
namespace-aware: [('A1', '40.00'), ('A2', '12.50')]

Root cause

In XML, an element's identity is its namespace URI plus local name. <order> in no namespace and <order> in urn:supplier:orders:v2 are different elements, and XPath 1.0's //order only matches the first. The code also treated "zero rows" as success, so a broken query looked like a quiet day.

The fix

• Query with explicit prefixes mapped to the namespace URIs your contract allows. Accept known versions deliberately (v1 and v2), rather than stripping namespaces.
• Validate the feed against the supplier's XSD before importing: an unexpected namespace fails validation with a clear error.
• Add sanity checks: alert when a feed that usually has thousands of rows has zero, or when the count drops by more than a threshold.
• Keep the supplier's sample files for every version in your test suite.

Trade-offs

• Stripping all namespaces (a common shortcut) makes queries work, but merges vocabularies and hides real contract changes.
• Supporting several namespace versions means a little more code, but it lets partners upgrade on their own schedule.

Detect it in production

• Metrics for rows imported per feed, with alerts on zero and on sudden drops.
• Log the root element's namespace and the schema version on every run.
• Contract tests that run the latest sample file from each partner.

Key points

• A default namespace changes every element's name; namespace-blind XPath returns nothing, without an error.
• Map prefixes to URIs in queries; validate against the contract.
• Treat "zero results" from a feed as an incident until proven otherwise.