Snowflake Engineering Blog

Actually, Snowflake doesn’t use Arctic for SQL codegen internally. They use a different model chained with mistral-large… and they do smoke the competition. https://medium.com/snowflake/1-1-3-how-snowflake-and-mistral-push-the-…

We have published some insights here. https://medium.com/snowflake/snowflake-arctic-cookbook-series-explorin…

smoke? it's the same as gpt4 https://medium.com/snowflake/1-1-3-how-snowflake-and-mistral-push-the-…

FWIW, I now analyze the Stack Overflow dumps on Snowflake https://medium.com/snowflake/how-to-load-the-stack-overflow-xml-dumps-…

Thanks! The Bazel posts stopped for a while after I left Google, but I'm now back at a different place where I'm working with Bazel once again. So you can expect the posts on this topic to gradually come back :) (For some context, here is one: https://medium.com/snowflake/addressing-bazel-ooms-38023b736fa2) Let's see if my plans to return to Europe in the next few years play out... Edit: Oh, and the few posts that look like Twitter threads (they are tagged like that) are hand-crafted and were an experiment. I first wrote the threads as blog posts, ensuring each paragraph fit in a tweet, and then copy/pasted them into Twitter. I wrote them as a blog post because I wanted to have the "unrolled" version in the canonical source for future reference, without relying on those unroll apps.

By truly open, we mean our releases use an OSI-recognized license (Apache-2) and we go beyond just model weights. Here are the things that we are open-sourcing: i) Open-Sourced Model Weights ii) Open-Sourced Fine-Tuning Pipeline. This is essentially the training code if you want to adapt this model to your use cases. This along with an associated cookbook will be released soon, so keep an eye on our repo for updates: https://github.com/Snowflake-Labs/snowflake-arctic/ iii) Open-Sourced Data Information: We trained on publicly available datasets, and we will share information on what these datasets are, how we processed and filtered them, composition of our datasets etc. They will be published as part of the cookbook series here: https://www.snowflake.com/en/data-cloud/arctic/cookbook/, shortly. iv) Open-Sourced Research: We will share all of our findings from our architecture studies, performance analysis etc. Again these will be published as part of the cookbook series. You can already see a few blogs covering MoE Architecture and Training Systems here: https://medium.com/snowflake/snowflake-arctic-cookbook-series-explorin… https://medium.com/snowflake/snowflake-arctic-cookbook-series-building… v) Pre-Training System information: We actually used the already open-sourced libraries DeepSpeed and Megatron-DeepSpeed for training optimizations and the model implementation for training…