Skip to content
Theorem Weekly TW markTWTheoremWeekly

The evidence behind the week in machine learning.

Model updates

Can Mistral Large 4 Reproduce Vulnerabilities Where Closed Models Refuse?

Mistral Large 4 achieves 82% on cyber index tests by reproducing and patching flaws, a task where frontier models often refuse to engage.

Key points

  • Mistral Large 4 scores 82% on the Artificial Analysis Cyber Index test.
  • The model uses a 1T parameter MoE architecture with 49B active parameters.
  • It combines Multi-head Latent Attention with a DeepSeek Sparse Attention indexer.
Can Mistral Large 4 Reproduce Vulnerabilities Where Closed Models Refuse?
Depth
82%Mistral Large 4 scores 82% on the Artificial Analysis Cyber Index test · reported
49BThe model uses a 1T parameter MoE architecture with 49B active parameters · reported
1 outlet · newest source 443h ago

Mistral released Mistral Large 4, also known as Le Chonk, on Hugging Face as a multimodal Mixture-of-Experts model. It accepts text and image inputs to generate text responses for various tasks.

Evidence: confirmedWelcome Le Chonk: Mistral Large 4 on Hugging Face

aifleet-huggingface

The evidence

Mistral Large 4 achieves an 82% score on the Artificial Analysis Cyber Index test, which requires reproducing real vulnerabilities in open-source software and then patching them. According to Mistral, several closed frontier models score near zero on this specific benchmark because they refuse to attempt the reproduction work.

Evidence: reportedWelcome Le Chonk: Mistral Large 4 on Hugging Face

aifleet-huggingface

Evidence: reportedWelcome Le Chonk: Mistral Large 4 on Hugging Face

aifleet-huggingface

The decision

Defenders often need to reproduce vulnerabilities to fix them, but closed-source API filters may block these defensive inputs as potential attacks. Mistral positions Large 4 for defensive security, noting that open-weight deployment keeps usage policy and access control under the user’s direct management during incidents.

Evidence: reportedWelcome Le Chonk: Mistral Large 4 on Hugging Face

aifleet-huggingface

What the evidence means

The model maintains high refusal rates on harmful prompts from JailbreakBench, StrongREJECT, and AgentHarm, exceeding other open-weight models. This suggests a balance between enabling defensive reproduction tasks and blocking malicious requests, though the model still declines harmful inputs.

What research readers should know

Mistral Large 4 tests the capability of open-weight models to perform verifiable security tasks that closed models refuse. The reported result is an 82% success rate on reproduce-and-patch benchmarks.reported A stated limitation is that the model was not explicitly trained for malware analysis or rule writing, though it performed well in internal tests.

Evidence: reportedWelcome Le Chonk: Mistral Large 4 on Hugging Face

aifleet-huggingface

Why it matters

  • Researchers can fine-tune smaller models using Mistral Large 4 traces for code patching, leveraging verifiable rewards in RL environments.
  • Open weights allow direct measurement of refusal behavior, removing reliance on provider descriptions for security auditing.
  • The architecture’s sparse attention selects up to 2,048 key positions, offering insights into efficient context handling for large windows.
  • Users gain control over deployment policies, preventing API access loss during active incident response scenarios.

Topics: Mistral, LLMs, cybersecurity, open weights, Mixture-of-Experts

Updated

Ask the article

Local keyword search of this article; no generated answers.

Which of these is confirmed?

Check the answer

Mistral released Mistral Large 4, also known as Le Chonk, on Hugging Face as a multimodal Mixture-of-Experts model.

The matched primary source and announcement wording support the confirmed label.

Did this article give you the evidence you needed?

No vote saved on this device.
Your choice stays on this device. No public tally.

Questions readers ask

How does Mistral Large 4 handle context efficiently?

It uses Multi-head Latent Attention to keep compressed KV representations and a DeepSeek Sparse Attention indexer to select key positions.

What is the parameter count for Mistral Large 4?

The model has about 1 trillion total parameters, with 49 billion active parameters per token.

Can researchers measure refusal behavior directly?

Yes, open weights let researchers measure the model’s refusal behavior directly instead of relying on the provider’s description.

Connected stories

This storyHOPE method prunes MoE experts using pairwise interaction scoresHOPE method prunes MoE experts uDoes Mistral Large 4 Open Weights Match Closed Models in Agentic Coding?Does Mistral Large 4 Open WeightDo LLM Parameter Counts Actually Predict Model Intelligence?Do LLM Parameter Counts ActuallyCan DeepSeek V3 Run on a MacBook Air Using Pure JAX Code?Can DeepSeek V3 Run on a MacBook

Reading receipt

  • Mistral Large 4 scores 82% on the Artificial Analysis Cyber Index test.
  • The model uses a 1T parameter MoE architecture with 49B active parameters.
  • It combines Multi-head Latent Attention with a DeepSeek Sparse Attention indexer.
Time spent: 0:00 · Depth: Full

Sources

  1. Hugging Face

Related stories

Background reading