These startups are chasing the next big thing in LLMs

Since the introduction of transformer neural networks in 2017, they have powered all major large language models but now face limitations in handling long text efficiently. Startups are exploring new architectures beyond transformers to improve LLM capabilities and reduce computational costs. This shift aims to address the high energy consumption and scaling challenges inherent in current transformer-based models.