Tokenization, the process by which many generative AI models make sense of data, is flawed in key ways.
    • seven_phone@lemmy.world
      2·
      9 days ago

      How big is the data set in German that the models can learn from, how big is the specific user base once set up and what language do the engineers building it speak?