Tokens vs Embeddings – what are they + how are they different?
Annie SextonTokens and embeddings are essential concepts to large language models (LLMs), and they both represent words – or meaning? Or something? What are they exactly, and how are they different?