Kevin Jesse

[LG] Chronos: Learning the Language of Time Series A F Ansari, L Stella, C Turkmen, X Zhang… [Amazon Web Services] (2024) arxiv.org/abs/2403.07815 - Chronos is a simple yet effective framework for pretrained probabilistic time series models, which tokenizes time series values using scaling and quantization into a fixed vocabulary and trains existing transformer-based language model architectures on these tokenized time series via the cross-entropy loss. - Chronos models were pretrained based on the T5 family on a large collection of publicly available datasets, complemented by a synthetic dataset generated via Gaussian processes to improve generalization. - In a comprehensive benchmark consisting of 42 datasets, Chronos models significantly outperform other methods on datasets that were part of the training corpus, and have comparable and occasionally superior zero-shot performance on new datasets, relative to methods that were trained specifically on them. - Chronos models can leverage time series data from diverse domains to improve zero-shot accuracy on unseen forecasting tasks, positioning pretrained models as a viable tool to greatly simplify forecasting pipelines. - A central aspect of Chronos is the integration of data augmentation strategies, including TSMix and KernelSynth, which address the inherent limitations of small training datasets in time series forecasting, enhancing model robustness and generalization. - Despite the apparent distinction between language models and time series forecasting models, Chronos challenges the necessity of time-series-specific modifications, demonstrating the potential for language model architectures to address a broad range of time series problems with minimal modifications. - By its very nature as a language model operating over a fixed vocabulary, Chronos can seamlessly integrate with future advancements in LLMs, making it an ideal candidate for further development as a generalist time series model. - Chronos follows a minimalist approach by tokenizing time series values into a fixed vocabulary and training existing language model architectures on these tokens without any time-series-specific design or features. It uses a categorical distribution to model the observations, performing regression via classification. - Chronos models are probabilistic by design and multiple realizations of the future can be obtained by autoregressively sampling from the predicted distribution. The categorical output distribution allows the model to learn arbitrary distributions including multimodal ones. - Chronos's accuracy, coupled with its relatively modest model size, positions it as a preferable alternative to larger, more computationally demanding models for zero-shot forecasting applications.