[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: Monitoring Pipelines Not Capturing OpenAI Data in Versions > v0.3.35 #7419

Open
samuelpetermoshi opened this issue Nov 27, 2024 · 1 comment

Comments

@samuelpetermoshi
Copy link
samuelpetermoshi commented Nov 27, 2024

Bug Report

Description:
After upgrading to OpenWebUI versions greater than v0.3.35 (currently testing on v0.4.6), I observed that data from built-in OpenAI models is no longer being sent to monitoring pipelines like Datadog and Langfuse. This issue does not occur in v0.3.35 and earlier versions.

Pipelines:

  1. Datadog: https://github.com/open-webui/pipelines/blob/main/examples/filters/datadog_filter_pipeline.py
  2. Langfuse: https://github.com/open-webui/pipelines/blob/main/examples/filters/langfuse_filter_pipeline.py

Steps to Reproduce:

  1. Host OpenWebUI on a local machine running version v0.3.35.
  2. Submit prompts to OpenWebUI using built-in OpenAI models.
  3. Verify that monitoring data is successfully sent to both Datadog and Langfuse.
  4. Upgrade OpenWebUI to a version greater than v0.3.35 (e.g., v0.4.6).
  5. Submit prompts to OpenWebUI using built-in OpenAI models.
  6. Check Datadog and Langfuse for the monitoring data.

Expected Behavior:
Monitoring data from built-in OpenAI models should continue to flow to Datadog and Langfuse across all versions, including those greater than v0.3.35.

Observed Behavior:

  • Monitoring data is not sent to Datadog or Langfuse when using OpenWebUI versions greater than v0.3.35.
  • This issue is specific to built-in OpenAI models, as data from other models or components may not be affected.

Environment Details:

  • Local Setup:
    • Version: v0.3.35
    • Monitoring Data Status: Sent successfully to Datadog and Langfuse.
  • On-Premise Setup:
    • Version: v0.4.6
    • Monitoring Data Status: Not sent to Datadog and Langfuse.

Attachments:
I have attached screenshots to demonstrate the issue:

  1. Monitoring pipelines capturing data in v0.3.35.
  2. No data captured in v0.4.6.
  3. Example traces in Langfuse and Datadog. (Note: Redacted data is intentional for privacy reasons.)

Impact:
This bug directly affects the ability to monitor and trace activities involving OpenAI data, which is critical for observability and debugging.

Request:
Please investigate this issue and confirm if this behavior is due to changes in how OpenWebUI handles OpenAI model data in newer versions. Let me know if further details or testing support is needed.

Thank You!
Thank you for your continuous efforts and updates to OpenWebUI. Looking forward to the fix!

The images show how it worked on v0.3.35
Screenshot 2024-11-27 at 21 54 28

Screenshot 2024-11-27 at 21 53 38 Screenshot 2024-11-27 at 21 55 06

cc: @ith , @maycszz , @SuvroBaner , @arnab55net

@rossm-mf
Copy link

We seem to experience the same however we run casperhansen/llama-3-70b-instruct-awq locally and this one seems to still work on Langfuse. Tried the latest Langfuse 2.9 then even the version rc 3.0.0 and update the langfuse filter pipeline

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants