Dev claim · claim-2326 · #56 of 154
“"In 2007, the Northern Hemisphere reached a record low in ice coverage and the Northwest Passage was opened.”
Claim tags (sorted stems; highlighted when a passage shares them)
- coverag
- hemispher
- ice
- low
- northern
- northwest
- open
- passag
- reach
- record
The model’s verdict
The Transformer retrained from the notebook (the 2024 weights were never saved), shown three ways. The first is how the notebook evaluated it.
Notebook protocol
Dev claims predicted in batches of 16, in file order, from the 2024 retrieved evidence, as the notebook does.
- Supports
- 23.5%
- Refutes
- 6.3%
- Not enough info
- 61.9%
- Disputed
- 8.2%
One claim at a time
The same model run on a single claim, which is how the Try-it page runs it.
- Supports
- 36.5%
- Refutes
- 11.0%
- Not enough info
- 39.5%
- Disputed
- 13.0%
With gold evidence
Fed the human-annotated evidence instead of retrieved passages. This is the 'val accuracy' the training loop reports.
- Supports
- 43.3%
- Refutes
- 12.6%
- Not enough info
- 29.8%
- Disputed
- 14.3%
Retrieved vs gold evidence
Gold passages were picked by the dataset’s annotators. Retrieved passages are what the TF-IDF rule returned from all 1.19M passages. Switch between the submitted 2024 output and the re-runs.
2024 submission
The passages the team actually retrieved and submitted in 2024 (their saved output file). The file records only passage ids, so scores are recomputed with the submission rule.
- Path
- not recorded
- Found
- 0 of 4 gold
- P
- 0.00
- R
- 0.00
- F
- 0.00
#1evidence-13342 It reached No.
- reach (shared with the claim)
- cos
- 0.420
- overlap
- 1.000
- score
- 1.420
- shared
- 1
#2evidence-223544 While the song did not reach the Billboard Hot 100, it reached No. 6 on the UK Singles Chart and was a top-ten hit in many other European countries.
- reach (shared with the claim)
- cos
- 0.420
- overlap
- 1.000
- score
- 1.420
- shared
- 1
#3evidence-763928 Kittur reached its zenith during the Mallasarja Desai.
- reach (shared with the claim)
- cos
- 0.420
- overlap
- 1.000
- score
- 1.420
- shared
- 1
#4evidence-101316 It was released as a single, with the album 's title track as the b-side, and reached # 49 on the Billboard Hot 100, but performed much better on the Adult Contemporary chart, reaching # 5.
- reach (shared with the claim)
- cos
- 0.420
- overlap
- 1.000
- score
- 1.420
- shared
- 1
#5evidence-205807 The 2014 Connecticut Open (formerly known as the New Haven Open at Yale) was a tennis tournament played on outdoor hard courts.
- open (shared with the claim)
- cos
- 0.383
- overlap
- 1.000
- score
- 1.383
- shared
- 1
#6evidence-18861 Dangerous is the eighth studio album by American recording artist Michael Jackson, released on November 26, 1991 by Epic Records.
- record (shared with the claim)
- cos
- 0.354
- overlap
- 1.000
- score
- 1.354
- shared
- 1
Gold evidence (4)
evidence-1017927 The 2007 melt season let to a minimum 39% below the 1979–2000 average, and for the first time in human memory, the fabled Northwest Passage opened completely.
- melt
- averag
- minimum
- northwest (shared with the claim)
- passag (shared with the claim)
- memori
- let
- fabl
- complet
evidence-956029 On August 21, 2007, the Northwest Passage became open to ships without the need of an icebreaker.
- becam
- need
- open (shared with the claim)
- northwest (shared with the claim)
- passag (shared with the claim)
- without
- august
- ship
evidence-29644 On September 14, 2007, the European Space Agency stated that ice loss that year had opened up the historically impassable passage, setting a new low of ice cover as seen in satellite measurements which went back to 1978.
- space
- ice (shared with the claim)
- satellit
- seen
- passag (shared with the claim)
- loss
- agenc
- measur
evidence-1079662 The extreme loss in 2007 rendered the passage "fully navigable".
- محمد
- fulli
- passag (shared with the claim)
- loss
- navig
- render
- extrem