8000 Refactor `cog predict` to support `--json` flag by aron · Pull Request #2404 · replicate/cog · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Refactor cog predict to support --json flag #2404

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 24 commits into from
Jun 13, 2025
Merged

Refactor cog predict to support --json flag #2404

merged 24 commits into from
Jun 13, 2025

Conversation

aron
Copy link
Contributor
@aron aron commented Jun 11, 2025 •
edited
Loading

This PR introduces cog predict --json that represents both input and output as JSON.

The intention behind this PR is to provide a better interface for machines to interface with cog predict. Such as running automated tests. In these cases trying to parse the existing unstable output is non-trivial. Having access to the full JSON prediction makes things much easier.

In order to do this correctly the PR also fixes a number of bugs with the --output behavior.

  • We now support passing a directory as an output value e.g. --output ./outputs
  • We now honor the --output flag for lists of files. Previously if a model returned a list of files they would all be output.<idx>.<ext> regardless of the --output flag.
  • We now error with a non-zero exit code if the prediction is not successful.

There are several ways to use cog predict with the --json flag:

  1. Passing a JSON string directly. 
    cog predict --json '{"prompt": "make me a sandwich"}'
  2. Passing a path to a JSON file. 
    cog predict --json @inputs.json
  3. Passing JSON via stdin, using the special @- reference. 
    cog predict --json @- < inputs.json

The output will be the entire prediction printed to stdout by default.

cog predict --json '{"prompt": "make me a sandwich"}'
{"status": "successful", "output": "blah blah", error: null}

The output can also be written to a file:

cog predict --json '{"prompt": "make me a sandwich"}' --output output.json
Output written to: output.json

michaeldwan
michaeldwan approved these changes Jun 11, 2025
View reviewed changes
Copy link
Member
@michaeldwan michaeldwan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is excellent, nice work! Just one suggestion about the stdin ux. And I'd like some tests around inputs and outputs someday, but this looks great for now. Nice job.

8W9aG
8W9aG approved these changes Jun 11, 2025
View reviewed changes
Copy link
Contributor
@8W9aG 8W9aG left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only minor comments from my side but otherwise looks good! Perhaps it could be good to add an integration test to make sure this doesn't get lost by other changes?

@aron
Copy link
Contributor Author
aron commented Jun 11, 2025

Thanks both, I'll fix the failing integration test and add a couple more before merging.

@8W9aG 8W9aG merged commit 0a49320 into main Jun 13, 2025
26 checks passed
@8W9aG 8W9aG deleted the cog-predict-json branch June 13, 2025 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@8W9aG 8W9aG 8W9aG approved these changes

@michaeldwan michaeldwan michaeldwan approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging th 396D is pull request may close these issues.
3 participants
@aron @michaeldwan @8W9aG
0