We estimate the stock of human-generated public text at around 300 trillion tokens. If trends continue, language models will fully utilize this stock between 2026 and 2032, or even earlier if intensely overtrained.
  • @[email protected]
    3•27 days ago

    Seems to me the more pressing issue is fixing hallucinations. Also see a barrier of getting access to all that info because of competition.