XML in Production · Data Correctness

The Discount That Became Zero

An optional discount element was sometimes empty, sometimes missing and sometimes nil. The code treated all three the same, and a pricing rule zeroed out discounts.

Create a free account to save this page, take notes and track your progress. Sign in

Scenario

Example team: a pricing service receives product updates. Each product may have <discount>: a percentage to apply. A partner's new system starts sending <discount/> (empty) for "no change" and omitting the element for "no discount".

What you see in production

Thousands of products lose their existing discounts overnight. Revenue looks great for a day; then the partner's account manager calls.

The naive code

discount = float(product.findtext("discount") or 0)   # missing, empty and nil all become 0

Reproduce it

Missing, empty and nilTry it yourself
from lxml import etree

XSI = "http://www.w3.org/2001/XMLSchema-instance"
feed = etree.fromstring(f"""<products xmlns:xsi="{XSI}">
  <product sku="A"><discount>15</discount></product>
  <product sku="B"/>
  <product sku="C"><discount/></product>
  <product sku="D"><discount xsi:nil="true"/></product>
</products>""".encode())

def naive(p):
    return float(p.findtext("discount") or 0)

def careful(p):
    el = p.find("discount")
    if el is None:
        return "keep current discount"           # element absent: no information
    if el.get(f"{{{XSI}}}nil") == "true":
        return "remove discount"                 # explicitly null
    if not (el.text or "").strip():
        return "reject: empty value"             # empty is not a number
    return f"set to {float(el.text)}%"

for p in feed:
    print(p.get("sku"), "| naive:", naive(p), "| careful:", careful(p))

Output

A | naive: 15.0 | careful: set to 15.0%
B | naive: 0.0 | careful: keep current discount
C | naive: 0.0 | careful: reject: empty value
D | naive: 0.0 | careful: remove discount

Root cause

XML has three different states for an optional value: absent, present but empty, and explicitly nil (xsi:nil="true", allowed when the schema says nillable="true"). findtext() returns None for absent and "" for empty, and the or 0 turned both, plus nil, into zero.

The fix

  • Agree the meaning of each state in the contract (often: absent = unchanged, nil = clear, empty = invalid).
  • Enforce it in the schema: give numbers a numeric type (xs:decimal rejects an empty string), and use nillable="true" where null is meaningful.
  • Parse with explicit branches, not or default.

Trade-offs

  • Strict schemas reject messages partners used to get away with; roll out with a reporting period first.
  • Distinguishing three states adds code, but it's the only way to support partial updates correctly.

Detect it in production

  • Track how many values change per import; alert on mass changes to the same field.
  • Count each state (absent, empty, nil) per partner.

Key points

  • Absent, empty and nil are different in XML; don't collapse them with or default.
  • Put the meaning in the contract and the schema.
  • Alert on mass updates that change one field everywhere.

Practice questions

How do you represent an explicit null in XML, and what does the schema need?

Answer

Use xsi:nil="true" on the element (with the XML Schema instance namespace), and declare the element nillable="true" in the XSD.

Why does an xs:decimal element reject <discount/>?

Answer

An empty string isn't in the lexical space of xs:decimal, so validation fails, which catches 'empty' values early instead of silently treating them as zero.

Open in the app to mark lessons complete, take notes and track your progress.

Was this page helpful?