8000 bug: Athena query fails to return results from query with timestamp comparison · Issue #12616 · localstack/localstack · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

bug: Athena query fails to return results from query with timestamp comparison #12616

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
1 task done
yoyoyeti opened this issue May 13, 2025 · 1 comment
Open
1 task done

bug: Athena query fails to return results from query with timestamp comparison #12616

yoyoyeti opened this issue May 13, 2025 · 1 comment
Assignees
@bentsku
Labels
aws:athena Amazon Athena status: in progress Currently being worked on type: bug Bug report

Comments

@yoyoyeti
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

When running a query such as this:
SELECT * FROM test_db.timestamp_test WHERE event_time < CAST('2025-05-27 00:00:00.000' AS TIMESTAMP);
it will fail and log an error that looks like this:

2025-05-13T19:07:05.599 ERROR --- [ Thread-80] l.p.c.s.athena.provider : Athena error for query "SELECT *
FROM test_db.timestamp_test
WHERE event_time < CAST('2025-05-18 19:07:05.497' AS TIMESTAMP)"
Traceback (most recent call last):
File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/pro/core/services/athena/provider.py.enc", line 36, in S
run_safe(lambda:resources.create_s3_bucket(F,s3_client=G));L=A[E];J=query_utils.execute_query(L,database=T,context=P,execution_parameters=A[R]);B.exec_results[A[D]]=J;LOG.debug('Athena result for query "%s":\n%s',A[E],J);U=convert_to_csv(J);M=f"{K}/results.csv";V=f"s3://{F}/{M}";H['ResultConfiguration']=ResultConfiguration(OutputLocation=V);G.put_object(Body=to_bytes(U),Bucket=F,Key=M)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/pro/core/services/athena/query_utils.py.enc", line 51, in execute_query
return execute_trino_query(A,database=B,prepared_statements=G)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/pro/core/utils/bigdata/bigdata_utils.py.enc", line 46, in execute_trino_query
def execute_trino_query(query,database=_A,prepared_statements=_A):A=execute_trino_queries([query],database=database,prepared_statements=prepared_statements);return A[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/pro/core/utils/bigdata/bigdata_utils.py.enc", line 68, in execute_trino_queries
for I in queries:J=H(I);B.append(J)
^^^^
File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/pro/core/utils/bigdata/bigdata_utils.py.enc", line 65, in H
raise Exception(f"Unable to get result from Trino server after {TRINO_STARTUP_QUERY_RETRIES} retries: {A}")
Exception: Unable to get result from Trino server after 15 retries: PrestoExternalError(type=EXTERNAL, name=HIVE_CANNOT_OPEN_SPLIT, message="Error opening Hive split s3a://test-bucket/data/9bd32a22-b5dc-4961-816f-5aa397d59f29.parquet (offset=0, length=1396): java.io.IOException: can not read class org.apache.parquet.format.ColumnIndex: Required field 'null_pages' was not present! Struct: ColumnIndex(null_pages:null, min_values:[59 D4 E7 CA 96 01 00 00], max_values:[59 D4 E7 CA 96 01 00 00], boundary_order:null)", query_id=20250513_190705_00010_q7qgi)

If no rows meet the condition, then no error is logged and no results are returned as expected.

It also returns the row without errors if I just do SELECT * FROM test_db.timestamp_test.

I'm also able to get the query to work the way I want if I do this:
SELECT * FROM test_db.timestamp_test WHERE from_unixtime(to_unixtime(event_time)) < CAST('2025-05-27 00:00:00.000' AS TIMESTAMP);. I need to be able to run the query without that workaround though.

Expected Behavior

I expect the timestamp comparison to work and the row's matching the condition to be returned without any errors. This is the case when using Athena in AWS.

How are you starting LocalStack?

With a docker-compose file

Steps To Reproduce

How are you starting localstack / Client commands

There's a lot of setup I needed to reproduce this, so I made a repo that I'm hoping will make things easier. https://github.com/yoyoyeti/localstack-bug-example

Localstack is started with docker-compose and several commands are run in fixtures/localstack/ready.d/ when it launches. Then repro.ts contains the code that run's the query. After running everything, I can see the error in the docker container's logs.

Environment

- OS: MacOS 15.4.1
- LocalStack version: 4.4.1.dev1
  LocalStack build date: 2025-05-08
  LocalStack build git hash: 0ca26ac03

Anything else?

No response
@yoyoyeti yoyoyeti added type: bug Bug report status: triage needed Requires evaluation by maintainers labels May 13, 2025
@localstack-bot
Copy link
Collaborator

Welcome to LocalStack! Thanks for reporting your first issue and our team will be working towards fixing the issue for you or reach out for more background information. We recommend joining our Slack Community for real-time help and drop a message to LocalStack Support if you are a licensed user! If you are willing to contribute towards fixing this issue, please have a look at our contributing guidelines.

@Anze1508 Anze1508 added aws:athena Amazon Athena status: backlog Triaged but not yet being worked on and removed status: triage needed Requires evaluation by maintainers labels May 14, 2025
@bentsku bentsku self-assigned this May 15, 2025
@bentsku bentsku added status: in progress Currently being worked on and removed status: backlog Triaged but not yet being worked on labels May 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bentsku bentsku

Labels
aws:athena Amazon Athena status: in progress Currently being worked on type: bug Bug report
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@yoyoyeti @bentsku @localstack-bot @Anze1508
0