Dev claim · claim-2300 · #59 of 154
“"Austria is today seeing its earliest snowfall in history with 30 to 40 centimetres already predicted in the mountains.”
Claim tags (sorted stems; highlighted when a passage shares them)
- alreadi
- austria
- centimetr
- earliest
- histori
- mountain
- predict
- see
- snowfal
- today
The model’s verdict
The Transformer retrained from the notebook (the 2024 weights were never saved), shown three ways. The first is how the notebook evaluated it.
Notebook protocol
Dev claims predicted in batches of 16, in file order, from the 2024 retrieved evidence, as the notebook does.
- Supports
- 59.8%
- Refutes
- 22.3%
- Not enough info
- 5.7%
- Disputed
- 12.2%
One claim at a time
The same model run on a single claim, which is how the Try-it page runs it.
- Supports
- 51.8%
- Refutes
- 25.3%
- Not enough info
- 15.1%
- Disputed
- 7.7%
With gold evidence
Fed the human-annotated evidence instead of retrieved passages. This is the 'val accuracy' the training loop reports.
- Supports
- 11.7%
- Refutes
- 2.6%
- Not enough info
- 80.6%
- Disputed
- 5.1%
Retrieved vs gold evidence
Gold passages were picked by the dataset’s annotators. Retrieved passages are what the TF-IDF rule returned from all 1.19M passages. Switch between the submitted 2024 output and the re-runs.
2024 submission
The passages the team actually retrieved and submitted in 2024 (their saved output file). The file records only passage ids, so scores are recomputed with the submission rule.
- Path
- not recorded
- Found
- 0 of 5 gold
- P
- 0.00
- R
- 0.00
- F
- 0.00
#1evidence-261567 aquae dulcis, Roux, 1901 see Lacrymaria pupula
- see (shared with the claim)
- cos
- 0.506
- overlap
- 1.000
- score
- 1.506
- shared
- 1
#2evidence-180929 in Námestovo, see Hviezdoslavovo námestie (Námestovo)
- see (shared with the claim)
- cos
- 0.506
- overlap
- 1.000
- score
- 1.506
- shared
- 1
#3evidence-823025 See CarbFix.
- see (shared with the claim)
- cos
- 0.506
- overlap
- 1.000
- score
- 1.506
- shared
- 1
#4evidence-419945 see Acourtia Berylsimpsonia Lophopappus Vernonanthura
- see (shared with the claim)
- cos
- 0.506
- overlap
- 1.000
- score
- 1.506
- shared
- 1
#5evidence-228808 see Arctophila Catabrosa Cleistogenes Diarrhena Disakisperma Festuca Glyceria Poa Puccinellia
- see (shared with the claim)
- cos
- 0.506
- overlap
- 1.000
- score
- 1.506
- shared
- 1
#6evidence-306140 For Ludwigshafen am Bodensee, see Bodman-Ludwigshafen.
- see (shared with the claim)
- cos
- 0.506
- overlap
- 1.000
- score
- 1.506
- shared
- 1
Gold evidence (5)
evidence-163344 From 1925 to 1929, the economy enjoyed a short high before nearly crashing[clarification needed] after Black Tuesday.
- nearli
- crash
- need
- short
- enjoy
- economi
- tuesday
- black
- high
evidence-612818 Austria lies between latitudes 46° and 49° N, and longitudes 9° and 18° E. It can be divided into five areas, the biggest being the Eastern Alps, which constitute 62% of the nation's total area.
- latitud
- area
- longitud
- constitut
- biggest
- n
- alp
- austria (shared with the claim)
- divid
evidence-208322 Although Austria is cold in the winter (−10 to 0 °C), summer temperatures can be relatively high, with average temperatures in the mid-20s and a highest temperature of 40.5 °C (105 °F) in August 2013.
- averag
- highest
- winter
- mid
- cold
- temperatur
- summer
- austria (shared with the claim)
- rel
evidence-776430 Spring is variable and autumn cool, with possible snowfalls already in November.
- variabl
- snowfal (shared with the claim)
- cool
- possibl
- novemb
- spring
- alreadi (shared with the claim)
- autumn
evidence-977782 This law of neutrality, passed in late October 1955 (and not the State Treaty itself), ensured that modern Austria would align with neither NATO nor the Soviet bloc, and is considered one of the reasons for Austria's delayed entry into the European Union in 1995.
- align
- austria (shared with the claim)
- neither
- nato
- bloc
- delay
- neutral
- ensur