XML in Production · Data Correctness
The Discount That Became Zero
An optional discount element was sometimes empty, sometimes missing and sometimes nil. The code treated all three the same, and a pricing rule zeroed out discounts.
Scenario
Example team: a pricing service receives product updates. Each product may have <discount>: a percentage to apply. A partner's new system starts sending <discount/> (empty) for "no change" and omitting the element for "no discount".
What you see in production
Thousands of products lose their existing discounts overnight. Revenue looks great for a day; then the partner's account manager calls.
The naive code
discount = float(product.findtext("discount") or 0) # missing, empty and nil all become 0
Reproduce it
from lxml import etree
XSI = "http://www.w3.org/2001/XMLSchema-instance"
feed = etree.fromstring(f"""<products xmlns:xsi="{XSI}">
<product sku="A"><discount>15</discount></product>
<product sku="B"/>
<product sku="C"><discount/></product>
<product sku="D"><discount xsi:nil="true"/></product>
</products>""".encode())
def naive(p):
return float(p.findtext("discount") or 0)
def careful(p):
el = p.find("discount")
if el is None:
return "keep current discount" # element absent: no information
if el.get(f"{{{XSI}}}nil") == "true":
return "remove discount" # explicitly null
if not (el.text or "").strip():
return "reject: empty value" # empty is not a number
return f"set to {float(el.text)}%"
for p in feed:
print(p.get("sku"), "| naive:", naive(p), "| careful:", careful(p))
Output
A | naive: 15.0 | careful: set to 15.0% B | naive: 0.0 | careful: keep current discount C | naive: 0.0 | careful: reject: empty value D | naive: 0.0 | careful: remove discount
Root cause
XML has three different states for an optional value: absent, present but empty, and explicitly nil (xsi:nil="true", allowed when the schema says nillable="true"). findtext() returns None for absent and "" for empty, and the or 0 turned both, plus nil, into zero.
The fix
- Agree the meaning of each state in the contract (often: absent = unchanged, nil = clear, empty = invalid).
- Enforce it in the schema: give numbers a numeric type (
xs:decimalrejects an empty string), and usenillable="true"where null is meaningful. - Parse with explicit branches, not
or default.
Trade-offs
- Strict schemas reject messages partners used to get away with; roll out with a reporting period first.
- Distinguishing three states adds code, but it's the only way to support partial updates correctly.
Detect it in production
- Track how many values change per import; alert on mass changes to the same field.
- Count each state (absent, empty, nil) per partner.
Key points
- Absent, empty and nil are different in XML; don't collapse them with
or default. - Put the meaning in the contract and the schema.
- Alert on mass updates that change one field everywhere.
Practice questions
How do you represent an explicit null in XML, and what does the schema need?
Answer
Use xsi:nil="true" on the element (with the XML Schema instance namespace), and declare the element nillable="true" in the XSD.
Why does an xs:decimal element reject <discount/>?
Answer
An empty string isn't in the lexical space of xs:decimal, so validation fails, which catches 'empty' values early instead of silently treating them as zero.
Open in the app to mark lessons complete, take notes and track your progress.