Search
Contact salesLog in to AI StudioLog in to AI Cloud

Blog

AI research and insights

SWE-rebench dataset: More than 21,000 verifiable tasks for SWE agents

Our AI R&D team announces the open-source release of the SWE-rebench dataset of more than 21,000 real-world, interactive software engineering tasks. For a detailed methodology and technical report, please see our accompanying paper on arXiv.

2025-06-10
1 min to read
AI research and insights

SWE-rebench: A continuously updated benchmark for SWE LLMs

Our AI R&D team presents SWE-rebench, a new benchmark for evaluating agentic LLMs on a continuously updated and decontaminated set of real-world software engineering tasks mined from real GitHub repositories. Our goal with this benchmark is to make evaluation of software engineering LLMs more transparent, reproducible and focused on core model capabilities.

2025-05-14
1 min to read
AI research and insights

Reasoning critics enable better parallel search for software engineering agents

In this short research blog post, we discuss how reinforcement learning-finetuned reasoning language models can be used as a better alternative to regression-based critics when performing parallel trajectory search in test time, and how to train such models to prioritize precision over recall.

2025-05-01
10 mins to read
AI research and insights

Kvax: Fast and easy-to-use FlashAttention implementation for JAX

Today, we’re open-sourcing Kvax, our FlashAttention implementation based on JAX. Designed for efficient training with long sequences, Kvax supports context parallelism and optimized computation of document masks. It outperforms many other FlashAttention implementations in long-context training with dense packing, achieving state-of-the-art performance.

2025-02-27
13 mins to read
AI research and insights

Scaling data collection for training software engineering agents

In this follow-up to our previous research blog post, we focus on the data collection process used to train our action generator and critic models. We’re also releasing two datasets on Hugging Face: nebius/SWE-bench-extra, containing 6,411 Issue/Pull-Request pairs, and nebius/SWE-agent-trajectories, featuring 80,036 software engineering agent trajectories, where an agent attempts to solve these issues.

2024-12-20
9 mins to read
AI research and insights

Leveraging training and search for better software engineering agents

In this blog post, we’ll share some of our recent findings on how search can enhance agentic systems for software engineering. Specifically, we’ll demonstrate how we’ve built an SWE-agent-based system that exclusively uses open-weight LLMs and achieves a 40.6% resolved rate on SWE-bench Verified.

2024-11-15
10 mins to read