8000 Codex ignores its system prompt after the first message · Issue #900 · openai/codex · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Codex ignores its system prompt after the first message #900

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
arnaudstiegler opened this issue May 12, 2025 · 8 comments
Open

Codex ignores its system prompt after the first message #900

arnaudstiegler opened this issue May 12, 2025 · 8 comments
Labels
bug Something isn't working

Comments

@arnaudstiegler
Copy link

What version of Codex is running?

0.1.2504301751

Which model were you using?

o3

What platform is your computer?

Darwin 24.4.0 arm64 arm

What steps can reproduce the bug?

`
╭──────────────────────────────────────────────────────────────╮
│ ● OpenAI Codex (research preview) v0.1.2504301751 │
╰──────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────────────────────╮
│ localhost session: db26dfe84d514d62bbafaa469d78c4a5 │
│ ↳ workdir: ~/atlas-act-cli │
│ ↳ model: o3 │
│ ↳ provider: openai │
│ ↳ approval: suggest │
╰──────────────────────────────────────────────────────────────╯
user
As per your dev guidelines, how should you edit files?

codex
According to the developer guidelines, any time you need to change or add code you should:

• Perform the edit with an apply_patch tool call, for example:

    {
      "cmd": ["apply_patch", "*** Begin Patch\n*** Update File: path/to/file.py\n@@\n- old line\n+ new line\n*** End Patch"]
    }

Key points:

    1. Always wrap changes between “*** Begin Patch” and “*** End Patch”.
    2. Specify the action (Add, Delete, Update, Rename) and the file path after the first marker line.
    3. Show only the minimal diff (- and + lines) needed for the fix.
    4. Let the system save the file; do not instruct the user to copy/paste it.

Using apply_patch keeps a clean git history and allows the sandbox to track and test your changes automatically.

user
What about grep?

codex
To search the repository you can invoke the shell tool with a grep (or ripgrep / rg) command.  A typical pattern looks like this (remember: the function call itself must appear in the commentary channel, never the analysis
channel).

Example:

    {"name": "shell",
     "arguments": {
       "command": ["grep", "-R", "-n", "search_term", "."]
     }}

Guideline reminders:

    1. Use the shell tool only from the commentary channel.
    2. Keep the command minimal and safe (e.g., don’t pipe to destructive commands).
    3. You can adjust flags (`-R`, `-n`, `--line-number`, etc.) or switch to `rg` if available for faster searches.
    4. The tool returns the command’s stdout/stderr so you can inspect results and decide what to do next.

`

What is the expected behavior?

No response

What do you see instead?

The system prompt clearly outlines that grep should not be used and only ripgrep should be used. Codex will answer correctly the question "Is grep allowed?" if it's the first message, but not for any subsequent message

Additional information

No response
@arnaudstiegler arnaudstiegler added the bug Something isn't working label May 12, 2025
@arnaudstiegler
Copy link
Author

@bolinfest I think this is the commit that introduces the regression: b323d10#diff-81c43c273518e6060612e02cf13bc1863785ee2316ce94ded66d624cf4621041L634-R692

@bolinfest
Copy link
Collaborator

@arnaudstiegler Based on what you pasted in the PR, I believe you are using the default version of the CLI, which is in TypeScript.

The PR that you linked to is purely a Rust change, so I do not believe it could be related to this. How did you make that connection?

@arnaudstiegler
Copy link
Author

You're right! I did observe the same behavior on the rust side, and I thought the reason was the change on the instruction logic:

  let instructions = if is_first_turn {
      sess.instructions.clone()
  } else {
      None
  };

I observe the same issue with both rust and the TS implementation

@bolinfest
Copy link
Collaborator

Looking at agent-loop.ts, it appears that the full instructions are always sent:

codex/codex-cli/src/utils/agent/agent-loop.ts

Line 730 in 55142e3

instructions: mergedInstructions,

codex/codex-cli/src/utils/agent/agent-loop.ts

Line 1112 in 55142e3

instructions: mergedInstructions,

so I think this is a mistake in the Rust code, though given that they are sent for every turn in the TypeScript, I'm surprised you're seeing this issue there...

@bolinfest
Copy link
Collaborator

Also, looking at #642 (which introduced the codex-rs folder to the repo), I think this issue with the instructions only being sent on the first turn in the Rust version has been present since the initial import:

codex/codex-rs/core/src/codex.rs

Lines 634 to 637 in 31d0d7a

let instructions = match prev_id {
Some(_) => None,
None => sess.instructions.clone(),
};

@bolinfest bolinfest mentioned this issue May 13, 2025
Merged
bolinfest added a commit that referenced this issue May 13, 2025
@bolinfest
fix: agent instructions were not being included when ~/.codex/instruc…
61b881d 
…tions.md was empty (#908)

I had seen issues where `codex-rs` would not always write files without
me pressuring it to do so, and between that and the report of
#900, I decided to look into this
further. I found two serious issues with agent instructions:

(1) We were only sending agent instructions on the first turn, but
looking at the TypeScript code, we should be sending them on every turn.

(2) There was a serious issue where the agent instructions were
frequently lost:

* The TypeScript CLI appears to keep writing `~/.codex/instructions.md`:
https://github.com/openai/codex/blob/55142e3e6caddd1e613b71bcb89385ce5cc708bf/codex-cli/src/utils/config.ts#L586
* If `instructions.md` is present, the Rust CLI uses the contents of it
INSTEAD OF the default prompt, even if `instructions.md` is empty:
https://github.com/openai/codex/blob/55142e3e6caddd1e613b71bcb89385ce5cc708bf/codex-rs/core/src/config.rs#L202-L203

The combination of these two things means that I have been using
`codex-rs` without these key instructions:
https://github.com/openai/codex/blob/main/codex-rs/core/prompt.md

Looking at the TypeScript code, it appears we should be concatenating
these three items every time (if they exist):

* `prompt.md`
* `~/.codex/instructions.md`
* nearest `AGENTS.md`

This PR fixes things so that:

* `Config.instructions` is `None` if `instructions.md` is empty
* `Payload.instructions` is now `&'a str` instead of `Option<&'a
String>` because we should always have _something_ to send
* `Prompt` now has a `get_full_instructions()` helper that returns a
`Cow<str>` that will always include the agent instructions first.
@bolinfest
Copy link
Collaborator

@arnaudstiegler if nothing else, this alerted me to a serious Rust issue, now fixed in #908, so thanks for that!

@arnaudstiegler
Copy link
Author

Thanks for the fix!

@arnaudstiegler
Copy link
Author

Looks like the issue is solved for the rust implementation, but the typescript implementation still has the same symptom (forgets system prompt after the first message). See screenshot (ran on latest main commit):

Image Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bolinfest @arnaudstiegler
0