Skip to content

Allow llama.cpp reasoning preservation flags - #1032

Draft
cmizzi wants to merge 2 commits into
docker:mainfrom
cmizzi:feat/allow-reasoning-preserve
Draft

cmizzi wants to merge 2 commits into
docker:mainfrom
cmizzi:feat/allow-reasoning-preserve

Conversation

@cmizzi

@cmizzi cmizzi commented Aug 4, 2026

Copy link
Copy Markdown

This PR adds --reasoning-preserve and --no-reasoning-preserve to the llama.cpp runtime flag allowlist.

These flags are supported by llama.cpp and are boolean-only options. They do not access files or external resources, so they fit the existing security model used for runtime flag validation. --reasoning-preserve is important for reasoning models because it keeps the reasoning trace in the conversation history, allowing prompt/KV cache reuse when combined with prompt caching.

Without this change, Docker Model Runner rejects the flags before llama.cpp is started.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

All reviewed changes are covered and no blocking issues remain.

Review effort: Lite
Findings: None

What changed in this PR

Adds llama.cpp reasoning-preservation flags to runtime validation, enabling reasoning trace retention for prompt/KV cache reuse.

Changes:

  • Allowlists --reasoning-preserve and --no-reasoning-preserve.
  • Adds validation and category test coverage.
File Description
pkg/​inference/​runtime_flags_test.go Tests runtime validation for both flags.
pkg/​inference/​runtime_flags_allowlist.go Adds the two flags to the llama.cpp allowlist.
pkg/​inference/​runtime_flags_allowlist_test.go Covers reasoning-category membership.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

No unresolved review issues remain.

Review effort: Lite
Findings: None

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

No unresolved issues were identified, and the changes include relevant test coverage.

Review effort: Lite
Findings: None

@ericcurtin

Copy link
Copy Markdown
Contributor

Good addition, but it now conflicts with main since #1065 touched the same allowlist map. Please rebase, nothing else to fix. Also, model-runner is being deprecated in favor of llmman, please open future PRs there instead. Marking as draft, please mark ready for review once rebased.

@ericcurtin
ericcurtin marked this pull request as draft September 26, 2026 14:21

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants