Siffer — Soil Type Code Parser¶

What is the Siffer field?¶

Every polygon in the Estonian soil map is assigned one or more siffer (šiffer) codes — alphanumeric labels that identify the dominant soil type(s) within that mapping unit. A single polygon can carry up to four siffer codes, semicolon-separated, listed in order of decreasing dominance:

Ko;D;LPe;LP

This example encodes four soil types:

Code	Estonian name	Approximate international equivalent
`Ko`	Korestikmaa	Skeletic Leptosol / shallow rocky soil
`D`	Deluviaalmuld	Colluvic Regosol / slope deposit soil
`LPe`	Leetjas-paepealne erosioonimuld	Eroded Albeluvisol on limestone
`LP`	Leetjas-paepealne muld	Albeluvisol on limestone

The siffer vocabulary comprises several thousand codes defined in the national soil classification system. The valid codes for the current dataset are maintained in updated_uniq_jan25_2026.csv.

Why the raw data needs repair¶

The soil map was digitised from analogue sheets by many different operators over several decades. This produced a wide range of encoding artefacts:

Mixed delimiters — commas, spaces, colons, or dashes appear where semicolons should be used (e.g. Ko Ko LP or Ko,LP instead of Ko;LP).

Erosion-degree annotations — numeric erosion-intensity classes appended to soil codes where they do not belong (e.g. E1, E(1;2), C3 variants of the base codes E and C).

OCR and transcription errors — character swaps, merged tokens, stray brackets that arose when analogue text was scanned or retyped.

Legacy abbreviations — older mapping rounds used slightly different code spellings that are no longer part of the current standard.

The parser applies a multi-step repair workflow before the grammar validation:

Whole-string lookup in ~700 curated full-match replacements
Erosion-degree stripping (E-type, C-type, and parenthesised numerics)
Colon-separated numeric pair removal (last resort)
Delimiter normalisation (all separators → ;)
Per-token character-level lookup (~200 entries)

Only after these steps is the string validated against the formal Arpeggio grammar.

Output fields¶

The parser returns 7 columns per soil polygon row:

Field	Type	Description
`siffer_1`	str	First (dominant) soil-type code, standardised. Empty if absent.
`siffer_2`	str	Second soil-type code. Empty if absent.
`siffer_3`	str	Third soil-type code. Empty if absent.
`siffer_4`	str	Fourth soil-type code. Empty if absent.
`n_siffers`	int	Number of soil-type codes found in this polygon (0–4).
`parse_ok_s`	bool	`True` if all codes were recognised by the grammar. Used in the map viewer error-review style together with `parse_ok_l` and `parse_ok_h`.
`parse_error`	str	Description of what could not be parsed. Empty on success.

Empty vs absent

siffer_1 through siffer_4 are populated sequentially. If a polygon has only two soil types, siffer_1 and siffer_2 carry the codes and siffer_3, siffer_4 are empty strings.

Worked example¶

Raw field value: "LkI LP"

Step	Result
Full-match lookup	no change
Erosion stripping	no change
Delimiter normalisation	`"LkI;LP"`
Grammar parse (`semicolon_separated`)	siffer_1=`LkI`, siffer_2=`LP`

Output: siffer_1="LkI", siffer_2="LP", n_siffers=2, parse_ok_s=True

The bracket form "LkI(LP)" reaches the same output through the grammar's pair_symbol rule, with no repair step needed. The parser records the codes only — the difference in the second component's share that the bracket and semicolon forms carry (see Rules) is not encoded in the output columns.

No deduplication

The parser does not collapse repeated codes. "Ko;Ko;LP" parses to siffer_1="Ko", siffer_2="Ko", siffer_3="LP", n_siffers=3. Where a repeated raw value does come out shorter, that is a curated entry in siffer_repair_rules_lookup_full_match, not a pipeline stage.

Parse coverage¶

Across the full dataset (~800 k polygon rows), the siffer repair workflow resolves the large majority of non-standard entries. Rows where parse_ok_s=False represent codes absent from the current reference vocabulary or unresolvable artefacts; these are flagged for manual review.

Reference — Valid Siffer Codes¶

The table below lists every valid siffer code recognised by the Arpeggio grammar. Two placeholder columns (Description, Soil Type Class) are provided for future annotation by domain experts. The table is auto-generated from the canonical vocabulary file and must be kept in sync via the generator script.

315 unique siffer codes (generated from updated_uniq_jan25_2026.csv)

Code	Description	Soil Type Class
`AG`
`AG1`
`AM`
`AM’`
`AM’’`
`AM’’al`
`AM’’’`
`AM’’’al`
`AM’’’d`
`AM’’’õ`
`Ag`
`Ar`
`ArG`
`ArG1`
`Arv`
`Av`
`B`
`BG`
`BGa`
`BGal`
`Bg`
`Bgal`
`C`
`D`
`D(g)`
`DG`
`DG1`
`DGal`
`Dg`
`Dgal`
`E`
`EI`
`EI(g)`
`EIg`
`Eg`
`Ek`
`Ek(g)`
`Ekg`
`Eo`
`Eo(g)`
`Eog`
`G`
`G1`
`GI`
`GI1`
`GI1a`
`GI1al`
`GI1d`
`GIa`
`GIal`
`GId`
`GIe`
`Gal`
`Gh1`
`Gh1al`
`Gh’`
`Gh’’`
`Gh’’a`
`Gh’’al`
`Gk`
`Gk1`
`Gka`
`Gkal`
`Gkr`
`Gkral`
`Gkrf`
`Gk’’’`
`Gn`
`Go`
`Go1`
`Go1a`
`Go1al`
`Go1d`
`Goa`
`GoaI`
`Goal`
`God`
`Gor`
`Goral`
`Gov`
`Gr`
`Gr1`
`K`
`K(g)`
`K(g)al`
`K(g)e`
`KI`
`KI(g)`
`KI(g)d`
`KI(g)e`
`KIa`
`KId`
`KIe`
`KIg`
`KIga`
`KIgal`
`KIgd`
`KIge`
`KIje`
`Kd`
`Ke`
`Kf`
`Kg`
`Kgal`
`Kgd`
`Kh`
`Khg`
`Kh’`
`Kh’g`
`Kh’’`
`Kh’’(g)`
`Kh’’g`
`Kje`
`Kk`
`Kkg`
`Ko`
`Ko(g)`
`Ko(g)al`
`Ko(g)d`
`Ko(g)e`
`Kod`
`Koe`
`Kog`
`Koga`
`Kogal`
`Kogd`
`Koje`
`Kor`
`Kor(g)`
`Korg`
`Kr`
`Kr(Ko)`
`Kr(g)`
`Krd`
`Kre`
`Krf`
`Krg`
`Krgf`
`Krje`
`K’`
`L`
`L(k)`
`L(k)I`
`L(k)I(g)`
`L(k)II`
`L(k)II(g)`
`L(k)III`
`L(k)IIIg`
`L(k)IIIgn`
`L(k)IIe`
`L(k)IIg`
`L(k)IIgn`
`L(k)IIn`
`L(k)Ie`
`L(k)Ig`
`L(k)Ign`
`L(k)g`
`L(k)gn`
`LG`
`LG1`
`LG1(n)`
`LG1a`
`LG1al`
`LG1n`
`LGn`
`LI`
`LI(g)`
`LII`
`LIII`
`LIIIg`
`LIIIgn`
`LIIIn`
`LIIg`
`LIIgn`
`LIIn`
`LIe`
`LIg`
`LIge`
`LIgn`
`LIn`
`LP`
`LP(g)`
`LP(g)al`
`LP(g)d`
`LP(g)e`
`LPG`
`LPGal`
`LPGn`
`LPd`
`LPe`
`LPg`
`LPga`
`LPgal`
`LPgd`
`LPge`
`LPgf`
`LPgn`
`LPje`
`Lg`
`Lgn`
`Liivik`
`Lk`
`Lk(g)`
`LkG`
`LkG(M)`
`LkGa`
`LkGal`
`LkGn`
`LkI`
`LkI(g)`
`LkI(g)d`
`LkI(g)e`
`LkI1`
`LkII`
`LkII(g)`
`LkII(g)e`
`LkIII`
`LkIII(g)`
`LkIIIg`
`LkIIIgn`
`LkIIIn`
`LkIIe`
`LkIIg`
`LkIIgn`
`LkIIje`
`LkIIn`
`LkId`
`LkIe`
`LkIg`
`LkIgal`
`LkIgd`
`LkIge`
`LkIgn`
`LkIje`
`LkIn`
`Lke`
`Lkg`
`Lkgd`
`Lkgn`
`Lkje`
`Lo`
`Ls`
`Lsg`
`Ma`
`Mal`
`Md`
`Meri`
`Mõ`
`M’`
`M’a`
`M’al`
`M’d`
`M’õ`
`M’’`
`M’’a`
`M’’al`
`M’’d`
`M’’õ`
`M’’’`
`M’’’a`
`M’’’al`
`M’’’d`
`M’’’õ`
`Piiritagune ala`
`Pp`
`PpG`
`Ppg`
`Pu`
`PuG`
`Pug`
`Rõ`
`R’`
`R’’`
`R’’’`
`R’’’õ`
`Sõ`
`S’`
`S’’`
`S’’õ`
`S’’’`
`S’’’al`
`S’’’õ`
`Tu`
`TuG`
`TuM`
`TuM’`
`TuM’’`
`TuM’’’`
`Tug`
`Tx`
`TxG`
`TxM`
`TxM’`
`TxM’’`
`TxM’’’`
`TxR`
`TxR’’`
`TxR’’’`
`TxS’`
`TxS’’`
`TxS’’’`
`Txg`
`Ty`
`TyG`
`TyM`
`Tyg`
`Tz`
`TzG`
`TzG1`
`TzM`
`TzM’`
`TzM’’`
`TzM’’’`
`Tzg`
`Veeala, asustus või määramata`