Skip to content
Climate Claim Checker

Dev claim · claim-2819 · #71 of 154

“More importantly, the OISM list only contains 39 scientists who specialise in climate science.”

Claim tags (sorted stems; highlighted when a passage shares them)

  • climat
  • contain
  • importantli
  • list
  • oism
  • scienc
  • scientist
  • specialis

Annotators’ label

Not enough info Run this claim live

The model’s verdict

The Transformer retrained from the notebook (the 2024 weights were never saved), shown three ways. The first is how the notebook evaluated it.

Notebook protocol

Dev claims predicted in batches of 16, in file order, from the 2024 retrieved evidence, as the notebook does.

Supports✗ differs from the label
Supports
59.5%
Refutes
35.5%
Not enough info
0.3%
Disputed
4.7%

One claim at a time

The same model run on a single claim, which is how the Try-it page runs it.

Supports✗ differs from the label
Supports
52.2%
Refutes
40.8%
Not enough info
4.1%
Disputed
2.8%

With gold evidence

Fed the human-annotated evidence instead of retrieved passages. This is the 'val accuracy' the training loop reports.

Supports✗ differs from the label
Supports
49.8%
Refutes
14.0%
Not enough info
20.5%
Disputed
15.6%

Retrieved vs gold evidence

Gold passages were picked by the dataset’s annotators. Retrieved passages are what the TF-IDF rule returned from all 1.19M passages. Switch between the submitted 2024 output and the re-runs.

2024 submission

The passages the team actually retrieved and submitted in 2024 (their saved output file). The file records only passage ids, so scores are recomputed with the submission rule.

Path
not recorded
Found
0 of 5 gold
P
0.00
R
0.00
F
0.00
  1. #1evidence-1030718

    anthropogenic climate science.

    • scienc (shared with the claim)
    • anthropogen
    • climat (shared with the claim)
    cos
    0.739
    overlap
    0.667
    score
    1.405
    shared
    2

Gold evidence (5)

  1. evidence-572386

    Ongoing experiments are conducted by more than 4,000 scientists from many nations.

    • experi
    • scientist (shared with the claim)
    • mani
    • conduct
    • nation
    • ongo
  2. evidence-596256

    A botanist, plant scientist or phytologist is a scientist who specialises in this field.

    • scientist (shared with the claim)
    • speciali
    • plant
    • field
    • botanist
  3. evidence-363528

    The list includes scientists from several specialities or disciplines.

    • special
    • sever
    • scientist (shared with the claim)
    • disciplin
    • list (shared with the claim)
    • includ
  4. evidence-1161865

    In 2001, Scientific American took a random sample "of 30 of the 1,400 signatories claiming to hold a Ph.D. in a climate-related science."

    • scienc (shared with the claim)
    • random
    • took
    • relat
    • hold
    • signatori
    • ph
    • climat (shared with the claim)
    • scientif
    • sampl
  5. evidence-676759

    Crudely extrapolating, the petition supporters include a core of about 200 climate researchers – a respectable number, though rather a small fraction of the climatological community.

    • crude
    • fraction
    • core
    • research
    • climatolog
    • though
    • respect
    • climat (shared with the claim)
    • rather
    • petit