8000 [Bug]: Wrong function params shown in docs for BM25 Content Filter · Issue #1086 · unclecode/crawl4ai · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[Bug]: Wrong function params shown in docs for BM25 Content Filter #1086

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
< 8000 div>
RogerSinghChugh opened this issue May 7, 2025 · 0 comments
Open

[Bug]: Wrong function params shown in docs for BM25 Content Filter #1086

RogerSinghChugh opened this issue May 7, 2025 · 0 comments
Labels
🐞 Bug Something isn't working 🩺 Needs Triage Needs attention of maintainers

Comments

@RogerSinghChugh
Copy link

crawl4ai version

0.6.2

Expected Behavior

The docs have an example where it is showcased how to use BM25 Content Filter.
This is a snippet from there:

bm25_filter = BM25ContentFilter(
    user_query="machine learning",
    bm25_threshold=1.2,
    use_stemming=True
)

Any program with this snippet should run normally.

Current Behavior

The current function definition doesn't have any param called use_stemming, instead it has the 'language' param.
This is the init function of the BM25 content filter:

    def __init__(
        self,
        user_query: str = None,
        bm25_threshold: float = 1.0,
        language: str = "english",
    )

Image

Is this reproducible?

Yes

Inputs Causing the Bug



Steps to Reproduce


1. Go to https://docs.crawl4ai.com/core/markdown-generation/
2. Scroll down to the 5.1 BM25ContentFilter section
3. There you can see the incorrect param usage snippet.


Code snippets




OS


Windows using WSL(Ubuntu 22.04.5)


Python version


3.10.12


Browser


No response


Browser version


No response


Error logs & Screenshots (if applicable)


Traceback (most recent call last):

File "/home/****///testing_scraping.py", line 32, in 

bm25_filter = BM25ContentFilter(

TypeError: BM25ContentFilter.init() got an unexpected keyword argument 'use_stemming'

      		
    

      
  





        



            
            

              
            

        

      


      
    








      

    



      

  
            

    

      
    

    


      

          @RogerSinghChugh
RogerSinghChugh




            added
      

  🐞 Bug

  Something isn't working

      

  🩺 Needs Triage

  Needs attention of maintainers

  labels


      May 7, 2025

    

  












  
  

    

      

  





  




  
  

              

  
    Sign up for free
    to join this conversation on GitHub.
    Already have an account?
    Sign in to comment


  



  






  
                  




    

  


      
  

    Assignees
  



        

    No one assigned







      


  


  

    Labels
  



    

      

    🐞 Bug

  Something isn't working
  

    🩺 Needs Triage

  Needs attention of maintainers








      

  

    

        

    Projects
  


        




    None yet




  



      


  

    
  

    Milestone
  


      No milestone





    
      
          


  

    

      
        

          
    

    Development
  




            
    
No branches or pull requests







      
    

  




      

    

  
      

  

    

      1 participant
    

    

        
          @RogerSinghChugh 
    

  






    

    




    






  

          

        



    



  





    
  


  


          



    



  

    

    

    





    

    

    

  



0