8000 feat: Render `vega` or `vega-lite` code blocks as interactive graphs by hydrosquall · Pull Request #327 · ucbepic/docetl · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat: Render vega or vega-lite code blocks as interactive graphs #327

New issue
Jump to bottom
Merged
merged 5 commits into from
Apr 4, 2025
Merged

feat: Render vega or vega-lite code blocks as interactive graphs #327

merged 5 commits into from
Apr 4, 2025

Conversation

hydrosquall
Copy link
Contributor
@hydrosquall hydrosquall commented Mar 24, 2025 •
edited
Loading

Motivation

Changes

  • Removed unused MarkdownCell code block (noticed the code is unused while figuring out how DocETL is wired)
  • Use vega-embed to render the graphs using latest vega/vega-lite packages. Ref docs for any questions about setup options. For better security since the Markdown fields might be untrusted if they are synthesized by an LLM, the graphs are drawn using the vega-interpreter mode rather than the speedy "eval" mode.
  • Added a bare-bones button for toggling between viewing as JSON or plain text. Rendering as plain text by default would be better for performance, but it's also convenient to be able to render graphs by default.

Testing

  • Here's a quick video clip of the graph in action when running on a 3 row sample! The cells work in both table and the expanded "view column output" modes.
2025-03-23_21-28-15.mp4

Notes

  • Open to any UX/UI feedback on how or if the graphs should be configured and/or the mode toggle should work. I just wanted to get something basic wired end-to-end.

  • Attached is an 8-row synthetic JSON dataset of vega-lite specifications, all using inlined data, which can be used for testing.

  • 8 good vega-lite-examples.json

  • 6 good, 2 bad vega-lite-examples.json -> use for testing error messages display when the input is invalid JSON.

@hydrosquall hydrosquall changed the title feat: Render vega or vega-lite code blocks as interactive graphs instead of plain text feat: Render vega or vega-lite code blocks as interactive graphs Mar 24, 2025
@hydrosquall hydrosquall force-pushed the cameron.yick/test-vega-render branch from 02cb966 to 9070bc0 Compare March 24, 2025 01:45
@hydrosquall hydrosquall force-pushed the cameron.yick/test-vega-render branch from 9070bc0 to 5962736 Compare March 24, 2025 01:46
@shreyashankar
Copy link
Collaborator

thank you so much for this--very exciting! will take a look today or tomorrow & merge :-)

@shreyashankar
Copy link
Collaborator

So sorry it's been taking a few days; I have paper deadlines next week so things are slower than expected here. But I will get to it tomorrow!

@hydrosquall
Copy link
Contributor Author
hydrosquall commented Mar 26, 2025 via email

@shreyashankar
Copy link
Collaborator

Just took a look; seems mostly great! A couple of questions:

Some cells don't render anything or a link to the vega lite editor...is it because the syntax is not correct? See screenshot below.
Screenshot 2025-03-26 at 8 00 07 AM

And, I get the following module not found:

 ⚠ ./node_modules/vega-canvas/build/vega-canvas.node.module.js
Module not found: Can't resolve 'canvas' in '/Users/shreyashankar/Documents/docetl/website/node_modules/vega-canvas/build'

I just ran npm install; is there something else I should install?

@hydrosquall
Copy link
Contributor Author

Hi @shreyashankar , there was a bug in that spec, dy does not exist, and the object did not contain a key.

I will fix the uploaded JSON Here is what we should have shown instead.

Let me see about displaying an error message in that case!

image

Regarding the second issue with canvas - were you able to get the graphs to appear still? canvas errors shouldn't appear, but I did recall hearing of some people having issues depending on what version of python was on their system and/or if a virtualenv was active.

vega/vega#3914 (comment)

There shouldn't be any additional installation steps necessary beyond the 3 packages I already added to the package.json (vega, vega-lite, and vega-embed).

@hydrosquall
Copy link
Contributor Author
hydrosquall commented Mar 27, 2025 •
edited
Loading
  1. I've added an simple tailwind-based error message when the inputted content doesn't parse as valid JSON.
image
  1. In the example JSON, there were 2 bad specs (both the layered bar you pointed out, and the histogram with inlined javascript.)

@shreyashankar
Copy link
Collaborator

thanks for the update! will take a look post April 1 :-)

@shreyashankar
Copy link
Collaborator

LGTM -- thank you!

@shreyashankar shreyashankar merged commit 40a8c0f into ucbepic:main Apr 4, 2025
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[feature] Allow table output cell to display data visualizations / charts using Vega/Vega-Lite
2 participants
@hydrosquall @shreyashankar
0