Description
Is your feature request related to a problem? Please describe.
The top-level entry points, notably sqlparse.parse(sql), use the singleton Lexer instance via FilterStack. Users that extend sqlparse (e.g. to parse SQL with a custom Lexer instance/subclass depending on the SQL's dialect) can't re-use top level functions/FilterStack, because they always use the singleton Lexer instance.
Describe the solution you'd like
I propose to make a subtle change that only affects the internals of FilterStack, adding a self.lexer = Lexer.get_default_instance() that users can override.
Describe alternatives you've considered
#717 attempts to solve this problem by adding a lexer parameter to top-level methods. My proposed change would not require any updates to public APIs/docs, like #715 does.
Additional context
Current usage with singleton lexer:
import sqlparse
sql = "SELECT id::!int FROM t"
statements = sqlparse.parse(sql)Note that parse(sql) boils down to:
def parse(sql):
stack = engine.FilterStack()
stack.enable_grouping()
return tuple(stack.run(sql)) # uses Lexer.get_default_instance() via lexer.tokens(...)New alternative extended usage (instead of unchanged top-level parse(sql)):
my_stack = engine.FilterStack()
my_stack.lexer = MyLexer() # extends sqlparse.lexer.Lexer
my_stack.enable_grouping()
statements = tuple(my_stack.run(sql))