What are SGlang and Marlin?

SGlang and Marlin are two optimization techniques that have made significant advancements in their respective fields.

SGlang:

• Released by LMC.org, SGlang is a new language aimed at speeding up various processes.
• It shows significant improvements, particularly for those utilizing different backends like OpenAI, Entropic, Gemini, and local models.
• It is particularly beneficial for fast execution of multiple tasks, especially in short strings communication between agents 1 2.

Marlin:

• Developed by RIAs from IST Austria, Marlin is a four-bit linear kernel optimization for language model (LM) inference.
• It achieves near-ideal 4x speed-ups for batch sizes of 16 to 32 tokens.
• Endorsed by notable experts, Marlin is seen as a significant breakthrough in the optimization cycles of deep learning models 1.

Both were released on the same day and have received high praise for their potential and effectiveness in boosting performance in their respective applications.