8000 Backporting LuaJIT upstream commits part 8 · Issue #9924 · tarantool/tarantool · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Backporting LuaJIT upstream commits part 8 #9924

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
14 tasks done
Tracked by #5483
Buristan opened this issue Apr 10, 2024 · 0 comments · Fixed by #10200
Closed
14 tasks done
Tracked by #5483

Backporting LuaJIT upstream commits part 8 #9924

Buristan opened this issue Apr 10, 2024 · 0 comments · Fixed by #10200
Assignees
Labels

Comments

@Buristan
Copy link
Collaborator
Buristan commented Apr 10, 2024

This ticket is an umbrella for all commits we planning to back-port in the next Q.
This list includes commits excluded from #9595 and may be updated later with some new.

general + x86/x64:

  • 65853056 Check for IR_HREF vs. IR_HREFK aliasing in non-nil store check.
  • f72c19e4 Maintain chain invariant in DCE.
  • 8a5e398c Avoid negation of signed integers in C that may hold INT*_MIN.
  • 4a22050d Prevent sanitizer warning in snap_restoredata().
  • d06beb04 Handle all types of errors during trace stitching.
  • b8b49bf3 Use generic trace error for OOM during trace stitching.
  • b2791179 Correct fix for stack check when recording BC_VARG.
  • d2fe2a6d Show name of NYI bytecode in -jv and -jdump.
  • 36b2962d FFI: Fix ffi.alignof() for reference types.
  • 899093a9 FFI: Fix sizeof expression in C parser for reference types.
  • 27a6fee8 FFI: Allow ffi.metatype() for typedefs with attributes.
  • 4c35a42d FFI: Fix ffi.metatype() for non-raw types.

Mach-O object files:

  • 3065c910 OSX/iOS/ARM64: Fix generation of Mach-O object files.
  • b98b3723 OSX/iOS/ARM64: Fix bytecode embedding in Mach-O object file.
mkokryashkin pushed a commit to tarantool/luajit that referenced this issue Apr 10, 2024
(cherry-picked from commit 9ebebc9)

Before the patch, there was a situation where `luaL_newstate`
could fail in main and the `argv[0]` could be used as a progname
in `l_message`. However, `argv[0]` is not guaranteed to be
non-NULL, so segmentation fault could occur. This patch fixes the
issue by using the predefined name in that case. Moreover, it
refactors the `l_message`, so now there is no need to pass
`pname` everywhere.

The patch is tested with the help of the mocking of
`luaL_newstate` by providing an error-injected implementation
of it and preloading it. For preload to work, the LuaJIT must
be built with dynamic build mode enabled. Corresponding flavor
is added to the CI.

The tarantool-c-tests target cannot be linked with the LuaJIT
library when it is built as shared. The test suite is disabled
for the dynamic build mode.

Part of tarantool/tarantool#9924
mkokryashkin pushed a commit to tarantool/luajit that referenced this issue Apr 11, 2024
(cherry-picked from commit 9ebebc9)

Before the patch, there was a situation where `luaL_newstate`
could fail in main and the `argv[0]` could be used as a progname
in `l_message`. However, `argv[0]` is not guaranteed to be
non-NULL, so the segmentation fault could occur. This patch fixes
the issue by using the predefined name in that case. Moreover, it
refactors the `l_message`, so now there is no need to pass
`pname` everywhere.

The patch is tested with the help of the mocking of
`luaL_newstate` by providing an error-injected implementation
of it and preloading it. For preload to work, the LuaJIT must
be built with dynamic build mode enabled. The corresponding
flavor is added to the CI.

The tarantool-c-tests target cannot be linked with the LuaJIT
library when it is built as shared. The test suite is disabled
for the dynamic build mode.

Since the Linux kernel 5.18-rc1 release, `argv` is forced to a
single empty string if it is empty [1], so the issue is not
reproducible on new kernels.

[1]: https://lore.kernel.org/all/20220201000947.2453721-1-keescook@chromium.org/

Part of tarantool/tarantool#9924
mkokryashkin pushed a commit to tarantool/luajit that referenced this issue Apr 11, 2024
(cherry-picked from commit 9ebebc9)

Before the patch, there was a situation where `luaL_newstate`
could fail in main and the `argv[0]` could be used as a progname
in `l_message`. However, `argv[0]` is not guaranteed to be
non-NULL, so the segmentation fault could occur. This patch fixes
the issue by using the predefined name in that case. Moreover, it
refactors the `l_message`, so now there is no need to pass
`pname` everywhere.

The patch is tested with the help of the mocking of
`luaL_newstate` by providing an error-injected implementation
of it and preloading it. For preload to work, the LuaJIT must
be built with dynamic build mode enabled. The corresponding
flavor is added to the CI.

The tarantool-c-tests target cannot be linked with the LuaJIT
library when it is built as shared. The test suite is disabled
for the dynamic build mode.

Since the Linux kernel 5.18-rc1 release, `argv` is forced to a
single empty string if it is empty [1], so the issue is not
reproducible on new kernels.

[1]: https://lore.kernel.org/all/20220201000947.2453721-1-keescook@chromium.org/

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue Apr 12, 2024
Reported by Yichun Zhang.

(cherry picked from commit b279117)

This patch is a follow-up to the commit
5f0a43a ("bugfix: fixed assertion
failure "lj_record.c:92: rec_check_slots: Assertion `nslots <= 250'
failed" found by stressing our edgelang compiler."), which is identical
to the commit e0388e6 ("Fix stack check
when recording BC_VARG.)" from the upstream. The error is raised too
late, when buffer overflow of `J->slot` has already occurred and data in
the `jit_State` structure is corrupted.

This patch moves the corresponding check before using the `J->slot`
buffer. The `J->maxslot` may overflow the buffer only in cases where the
amount of the vararg results is unknown. The check is used only in this
case since the trace recording for the undefined-on-trace varargs is not
yet implemented for an unknown amount of varargs.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Buristan added a commit to tarantool/luajit that referenced this issue Apr 19, 2024
This option enables table bump optimization if sink optimization is
enabled. The table bump optimization patches the bytecodes with a table
allocation on the trace recording if the recorded trace exceeds the size
of the allocated table. This optimization still has some bugs, so it is
disabled by default. For more details, see the comment in
<CMakeLists.txt>.

Needed for tarantool/tarantool#9924
Buristan added a commit to tarantool/luajit that referenced this issue Apr 19, 2024
This flavor enables the LUAJIT_ENABLE_TABLE_BUMP option to test table
bump optimization.

Needed for tarantool/tarantool#9924
Buristan added a commit to tarantool/luajit that referenced this issue Apr 19, 2024
Now information about the abort of the trace is saved in the
`abort_reason` field of the corresponding structure. The
`jit.parse.finish()` returns now the second table containing aborted
traces. Each table key is a trace number containing an array of
potentially traces with this number, which was aborted.

Needed for tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue Apr 19, 2024
Thanks to Sergey Kaplun and Peter Cawley.

(cherry picked from commit d06beb0)

This commit is a follow-up for the commit
1b82160 ("Throw any errors before stack
changes in trace stitching."). The patch prepends failures for the
specific error to be thrown. Nevertheless, the error may be thrown due
to retrying trace recording in the case when table bump optimization
is enabled or when OOM is observed during reallocation of the snapshot
or IR buffers.

This patch adds the corresponding protected frame and rethrows the error
after a fixup of the stack.

This patch also tests the correctness of copying the error message to
the top of the stack to get a valid "abort" reason in the `jit.dump`
utility.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue Apr 19, 2024
Thanks to Sergey Kaplun.

(cherry picked from commit b8b49bf)

The previous commit doesn't handle the case when the error code is
`LUA_ERRMEM`. This patch adds a workaround by using the generic error
message.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue Apr 19, 2024
Thanks to Sergey Kaplun and Peter Cawley.

(cherry picked from commit d06beb0)

This commit is a follow-up for the commit
1b82160 ("Throw any errors before stack
changes in trace stitching."). The patch prepends failures for the
specific error to be thrown. Nevertheless, the error may be thrown due
to retrying trace recording in the case when table bump optimization
is enabled or when OOM is observed during reallocation of the snapshot
or IR buffers.

This patch adds the corresponding protected frame and rethrows the error
after a fixup of the stack.

This patch also tests the correctness of copying the error message to
the top of the stack to get a valid "abort" reason in the `jit.dump`
utility.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue Apr 19, 2024
Thanks to Sergey Kaplun.

(cherry picked from commit b8b49bf)

The previous commit doesn't handle the case when the error code is
`LUA_ERRMEM`. This patch adds a workaround by using the generic error
message.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue Apr 19, 2024
Thanks to Sergey Kaplun.

(cherry picked from commit b8b49bf)

The previous commit doesn't handle the case when the error code is
`LUA_ERRMEM`. This patch adds a workaround by using the generic error
message.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue Apr 22, 2024
Thanks to Sergey Kaplun and Peter Cawley.

(cherry picked from commit d06beb0)

This commit is a follow-up for the commit
1b82160 ("Throw any errors before stack
changes in trace stitching."). The patch prepends failures for the
specific error to be thrown. Nevertheless, the error may be thrown due
to retrying trace recording in the case when table bump optimization
is enabled or when OOM is observed during reallocation of the snapshot
or IR buffers.

This patch adds the corresponding protected frame and rethrows the error
after a fixup of the stack.

This patch also tests the correctness of copying the error message to
the top of the stack to get a valid "abort" reason in the `jit.dump`
utility.

Also, this patch fixes a non-ASCII space character in the comment for
<lj-720-errors-before-stitch.test.lua>.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue Apr 22, 2024
Thanks to Sergey Kaplun.

(cherry picked from commit b8b49bf)

The previous commit doesn't handle the case when the error code is
`LUA_ERRMEM`. This patch adds a workaround by using the generic error
message.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue Apr 24, 2024
Thanks to Peter Cawley.

(cherry picked from commit 6585305)

The `lj_opt_fwd_wasnonnil()` skips the check for HREF and HREFK that may
alias. Hence, the guard for the non-nil value may be skipped, and the
`__newindex` metamethod call is omitted too.

This patch adds the aforementioned check for different reference types
(HREF vs. HREFK), which were not detected by the previous analysis.
Also, the helper macro `irt_isp32()` is introduced to check that the IR
type is `IRT_P32` (KSLOT type).

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue Apr 24, 2024
Reported by Yichun Zhang.

(cherry picked from commit b279117)

This patch is a follow-up to the commit
5f0a43a ("bugfix: fixed assertion
failure "lj_record.c:92: rec_check_slots: Assertion `nslots <= 250'
failed" found by stressing our edgelang compiler."), which is identical
to the commit e0388e6 ("Fix stack check
when recording BC_VARG.)" from the upstream. The error is raised too
late, when buffer overflow of `J->slot` has already occurred and data in
the `jit_State` structure is corrupted.

This patch moves the corresponding check before using the `J->slot`
buffer. The `J->maxslot` may overflow the buffer only in cases where the
amount of the vararg results is unknown. The check is used only in this
case since the trace recording for the undefined-on-trace varargs is not
yet implemented for an unknown amount of varargs.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924

Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
(cherry picked from commit 19dee09)
Buristan pushed a commit to tarantool/luajit that referenced this issue Apr 24, 2024
Reported by Yichun Zhang.

(cherry picked from commit b279117)

This patch is a follow-up to the commit
5f0a43a ("bugfix: fixed assertion
failure "lj_record.c:92: rec_check_slots: Assertion `nslots <= 250'
failed" found by stressing our edgelang compiler."), which is identical
to the commit e0388e6 ("Fix stack check
when recording BC_VARG.)" from the upstream. The error is raised too
late, when buffer overflow of `J->slot` has already occurred and data in
the `jit_State` structure is corrupted.

This patch moves the corresponding check before using the `J->slot`
buffer. The `J->maxslot` may overflow the buffer only in cases where the
amount of the vararg results is unknown. The check is used only in this
case since the trace recording for the undefined-on-trace varargs is not
yet implemented for an unknown amount of varargs.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924

Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Buristan pushed a commit to tarantool/luajit that referenced this issue Apr 24, 2024
Reported by Yichun Zhang.

(cherry picked from commit b279117)

This patch is a follow-up to the commit
5f0a43a ("bugfix: fixed assertion
failure "lj_record.c:92: rec_check_slots: Assertion `nslots <= 250'
failed" found by stressing our edgelang compiler."), which is identical
to the commit e0388e6 ("Fix stack check
when recording BC_VARG.)" from the upstream. The error is raised too
late, when buffer overflow of `J->slot` has already occurred and data in
the `jit_State` structure is corrupted.

This patch moves the corresponding check before using the `J->slot`
buffer. The `J->maxslot` may overflow the buffer only in cases where the
amount of the vararg results is unknown. The check is used only in this
case since the trace recording for the undefined-on-trace varargs is not
yet implemented for an unknown amount of varargs.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924

Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
(cherry picked from commit 19dee09)
Buristan added a commit to tarantool/luajit that referenced this issue May 13, 2024
This option enables table bump optimization if sink optimization is
enabled. The table bump optimization patches the bytecodes with a table
allocation on the trace recording if the recorded trace exceeds the size
of the allocated table. This optimization still has some bugs, so it is
disabled by default. For more details, see the comment in
<CMakeLists.txt>.

Needed for tarantool/tarantool#9924
Buristan added a commit to tarantool/luajit that referenced this issue May 13, 2024
This flavor enables the LUAJIT_ENABLE_TABLE_BUMP option to test table
bump optimization.

Needed for tarantool/tarantool#9924
Buristan added a commit to tarantool/luajit that referenced this issue May 13, 2024
Now information about the abort of the trace is saved in the
`abort_reason` field of the corresponding structure. The
`jit.parse.finish()` returns now the second table containing aborted
traces. Each table key is a trace number containing an array of
potentially traces with this number, which was aborted.

Needed for tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue May 13, 2024
Thanks to Sergey Kaplun and Peter Cawley.

(cherry picked from commit d06beb0)

This commit is a follow-up for the commit
1b82160 ("Throw any errors before stack
changes in trace stitching."). The patch prepends failures for the
specific error to be thrown. Nevertheless, the error may be thrown due
to retrying trace recording in the case when table bump optimization
is enabled or when OOM is observed during reallocation of the snapshot
or IR buffers.

This patch adds the corresponding protected frame and rethrows the error
after a fixup of the stack.

This patch also tests the correctness of copying the error message to
the top of the stack to get a valid "abort" reason in the `jit.dump`
utility.

Also, this patch fixes a non-ASCII space character in the comment for
<lj-720-errors-before-stitch.test.lua>.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue May 13, 2024
Thanks to Sergey Kaplun.

(cherry picked from commit b8b49bf)

The previous commit doesn't handle the case when the error code is
`LUA_ERRMEM`. This patch adds a workaround by using the generic error
message.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue May 13, 2024
Thanks to Peter Cawley.

(cherry picked from commit 6585305)

The `lj_opt_fwd_wasnonnil()` skips the check for HREF and HREFK that may
alias. Hence, the guard for the non-nil value may be skipped, and the
`__newindex` metamethod call is omitted too.

This patch adds the aforementioned check for different reference types
(HREF vs. HREFK), which were not detected by the previous analysis.
Also, the helper macro `irt_isp32()` is introduced to check that the IR
type is `IRT_P32` (KSLOT type).

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue May 17, 2024
Thanks to Peter Cawley.

(cherry picked from commit f72c19e)

Instructions with strong guards that are sometimes emitted with a guard
and sometimes emitted without a guard (like HREFK, CONV, or SLOAD) may
be eliminated from the IR chain and replaced with the NOP IR. If the
next IR of the same kind on the trace is not eliminated, it may
reference the IR NOP instead of an instruction of the same type. This
may lead to the corresponding assertion failure in the `rec_check_ir()`.

This patch unconditionally links the IRs during chain maintenance in
DCE.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue Jun 26, 2024
Suggested by Sergey Kaplun.

(cherry picked from commit d2fe2a6)

This patch replaces the number of NYI bytecodes that can't be compiled
with their names in the `jit.dump()` and -jv outputs. Since the
functionality is the same, only `jit.dump()` is tested as most popular.

Sergey Kaplun:
* added the description and the test for the feature

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue Jul 1, 2024
Suggested by Sergey Kaplun.

(cherry picked from commit d2fe2a6)

This patch replaces the numeric value of NYI bytecodes that can't be
compiled with their names in the `jit.dump()` and -jv outputs. Since the
functionality is the same, only `jit.dump()` is tested as most popular.

Sergey Kaplun:
* added the description and the test for the feature

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue Jul 1, 2024
Reported by minoki.
Recent C compilers 'take advantage' of the undefined behavior.
This completely changes the meaning of expressions like (k == -k).

(cherry picked from commit 8a5e398)

This patch changes all possibly dangerous -x operations on integers to
the corresponding two's complement. Also, it removes all related UBSAN
suppressions, since they are fixed.

Also, this patch limits the `bit.tohex()` result by 254 characters.

There is no testcase for `strscan_oct()`, `strscan_dec()` or/and
`STRSCAN_U32` format since first the unary minus is parsed first and
only after the number itself is parsed during parsing C syntax. So the
error is raised in `cp_expr_prefix()` instead. For parsing the exponent
header, there is no testcase, since the power is limited by
`STRSCAN_MAXEXP`.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Relates to tarantool/tarantool#8473
Buristan pushed a commit to tarantool/luajit that referenced this issue Jul 4, 2024
Thanks to Sergey Kaplun.

(cherry picked from commit 4a22050)

When saving FPR registers during while from a trace and restoring data
from a snapshot, UB sanitizer produces the following warning:
| lj_snap.c:804:32: runtime error: index 23 out of bounds for type 'intptr_t [16]'

due to indexing `ex->gpr` with a fpr register, whose number is >=
`RID_MAX_GPR`. The situation itself is harmless since this is read from
`spill[256]` array and is rewritten in the next if branch.

This patch fixes the out-of-bounds access to read from `ex->gpr` only
conditionally. Also, it removes the corresponding UBSAN suppression.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Relates to tarantool/tarantool#8473
Buristan pushed a commit to tarantool/luajit that referenced this issue Jul 4, 2024
Reported by minoki.
Recent C compilers 'take advantage' of the undefined behavior.
This completely changes the meaning of expressions like (k == -k).

(cherry picked from commit 8a5e398)

This patch changes all possibly dangerous -x operations on integers to
the corresponding two's complement. Also, it removes all related UBSAN
suppressions, since they are fixed.

Also, this patch limits the `bit.tohex()` result by 254 characters.

There is no testcase for `strscan_oct()`, `strscan_dec()` or/and
`STRSCAN_U32` format since first the unary minus is parsed first and
only after the number itself is parsed during parsing C syntax. So the
error is raised in `cp_expr_prefix()` instead. For parsing the exponent
header, there is no testcase, since the power is limited by
`STRSCAN_MAXEXP`.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Relates to tarantool/tarantool#8473
Buristan pushed a commit to tarantool/luajit that referenced this issue Jul 4, 2024
Thanks to Sergey Kaplun.

(cherry picked from commit 4a22050)

When saving FPR registers during while from a trace and restoring data
from a snapshot, UB sanitizer produces the following warning:
| lj_snap.c:804:32: runtime error: index 23 out of bounds for type 'intptr_t [16]'

due to indexing `ex->gpr` with a fpr register, whose number is >=
`RID_MAX_GPR`. The situation itself is harmless since this is read from
`spill[256]` array and is rewritten in the next if branch.

This patch fixes the out-of-bounds access to read from `ex->gpr` only
conditionally. Also, it removes the corresponding UBSAN suppression.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Relates to tarantool/tarantool#8473

Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
(cherry picked from commit 4eaacf8)
Buristan pushed a commit to tarantool/luajit that referenced this issue Jul 4, 2024
Reported by minoki.
Recent C compilers 'take advantage' of the undefined behavior.
This completely changes the meaning of expressions like (k == -k).

(cherry picked from commit 8a5e398)

This patch changes all possibly dangerous -x operations on integers to
the corresponding two's complement. Also, it removes all related UBSAN
suppressions, since they are fixed.

Also, this patch limits the `bit.tohex()` result by 254 characters.

There is no testcase for `strscan_oct()`, `strscan_dec()` or/and
`STRSCAN_U32` format since first the unary minus is parsed first and
only after the number itself is parsed during parsing C syntax. So the
error is raised in `cp_expr_prefix()` instead. For parsing the exponent
header, there is no testcase, since the power is limited by
`STRSCAN_MAXEXP`.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Relates to tarantool/tarantool#8473

Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
(cherry picked from commit 78f4de4)
Buristan pushed a commit to tarantool/luajit that referenced this issue Jul 4, 2024
Thanks to Sergey Kaplun.

(cherry picked from commit 4a22050)

When saving FPR registers during while from a trace and restoring data
from a snapshot, UB sanitizer produces the following warning:
| lj_snap.c:804:32: runtime error: index 23 out of bounds for type 'intptr_t [16]'

due to indexing `ex->gpr` with a fpr register, whose number is >=
`RID_MAX_GPR`. The situation itself is harmless since this is read from
`spill[256]` array and is rewritten in the next if branch.

This patch fixes the out-of-bounds access to read from `ex->gpr` only
conditionally. Also, it removes the corresponding UBSAN suppression.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Relates to tarantool/tarantool#8473

Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Buristan pushed a commit to tarantool/luajit that referenced this issue Jul 4, 2024
Reported by minoki.
Recent C compilers 'take advantage' of the undefined behavior.
This completely changes the meaning of expressions like (k == -k).

(cherry picked from commit 8a5e398)

This patch changes all possibly dangerous -x operations on integers to
the corresponding two's complement. Also, it removes all related UBSAN
suppressions, since they are fixed.

Also, this patch limits the `bit.tohex()` result by 254 characters.

There is no testcase for `strscan_oct()`, `strscan_dec()` or/and
`STRSCAN_U32` format since first the unary minus is parsed first and
only after the number itself is parsed during parsing C syntax. So the
error is raised in `cp_expr_prefix()` instead. For parsing the exponent
header, there is no testcase, since the power is limited by
`STRSCAN_MAXEXP`.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Relates to tarantool/tarantool#8473

Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Buristan pushed a commit to tarantool/luajit that referenced this issue Jul 4, 2024
Thanks to Sergey Kaplun.

(cherry picked from commit 4a22050)

When saving FPR registers during while from a trace and restoring data
from a snapshot, UB sanitizer produces the following warning:
| lj_snap.c:804:32: runtime error: index 23 out of bounds for type 'intptr_t [16]'

due to indexing `ex->gpr` with a fpr register, whose number is >=
`RID_MAX_GPR`. The situation itself is harmless since this is read from
`spill[256]` array and is rewritten in the next if branch.

This patch fixes the out-of-bounds access to read from `ex->gpr` only
conditionally. Also, it removes the corresponding UBSAN suppression.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Relates to tarantool/tarantool#8473

Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
(cherry picked from commit 4eaacf8)
Buristan pushed a commit to tarantool/luajit that referenced this issue Jul 4, 2024
Reported by minoki.
Recent C compilers 'take advantage' of the undefined behavior.
This completely changes the meaning of expressions like (k == -k).

(cherry picked from commit 8a5e398)

This patch changes all possibly dangerous -x operations on integers to
the corresponding two's complement. Also, it removes all related UBSAN
suppressions, since they are fixed.

Also, this patch limits the `bit.tohex()` result by 254 characters.

There is no testcase for `strscan_oct()`, `strscan_dec()` or/and
`STRSCAN_U32` format since first the unary minus is parsed first and
only after the number itself is parsed during parsing C syntax. So the
error is raised in `cp_expr_prefix()` instead. For parsing the exponent
header, there is no testcase, since the power is limited by
`STRSCAN_MAXEXP`.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9924
Relates to tarantool/tarantool#8473

Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
(cherry picked from commit 78f4de4)
Buristan pushed a commit to tarantool/luajit that referenced this issue Jul 4, 2024
Suggested by Sergey Kaplun.

(cherry picked from commit d2fe2a6)

This patch replaces the numeric value of NYI bytecodes that can't be
compiled with their names in the `jit.dump()` and -jv outputs. Since the
functionality is the same, only `jit.dump()` is tested as most popular.

Sergey Kaplun:
* added the description and the test for the feature

Part of tarantool/tarantool#9924
Buristan pushed a commit to tarantool/luajit that referenced this issue Jul 4, 2024
Suggested by Sergey Kaplun.

(cherry picked from commit d2fe2a6)

This patch replaces the numeric value of NYI bytecodes that can't be
compiled with their names in the `jit.dump()` and -jv outputs. Since the
functionality is the same, only `jit.dump()` is tested as most popular.

Sergey Kaplun:
* added the description and the test for the feature

Part of tarantool/tarantool#9924

Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
(cherry picked from commit 01f4586)
Buristan pushed a commit to tarantool/luajit that referenced this issue Jul 4, 2024
Suggested by Sergey Kaplun.

(cherry picked from commit d2fe2a6)

This patch replaces the numeric value of NYI bytecodes that can't be
compiled with their names in the `jit.dump()` and -jv outputs. Since the
functionality is the same, only `jit.dump()` is tested as most popular.

Sergey Kaplun:
* added the description and the test for the feature

Part of tarantool/tarantool#9924

Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
(cherry picked from commit 01f4586)
Buristan pushed a commit to tarantool/luajit that referenced this issue Jul 4, 2024
Suggested by Sergey Kaplun.

(cherry picked from commit d2fe2a6)

This patch replaces the numeric value of NYI bytecodes that can't be
compiled with their names in the `jit.dump()` and -jv outputs. Since the
functionality is the same, only `jit.dump()` is tested as most popular.

Sergey Kaplun:
* added the description and the test for the feature

Part of tarantool/tarantool#9924

Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Buristan added a commit to Buristan/tarantool that referenced this issue Jul 5, 2024
* Correct fix for stack check when recording BC_VARG.
* test: remove inline suppressions of _TARANTOOL
* FFI: Fix ffi.alignof() for reference types.
* FFI: Fix sizeof expression in C parser for reference types.
* FFI: Allow ffi.metatype() for typedefs with attributes.
* FFI: Fix ffi.metatype() for non-raw types.
* Maintain chain invariant in DCE.
* build: introduce option LUAJIT_ENABLE_TABLE_BUMP
* ci: add tablebump flavor for exotic builds
* test: allow `jit.parse` to return aborted traces
* Handle all types of errors during trace stitching.
* Use generic trace error for OOM during trace stitching.
* Check for IR_HREF vs. IR_HREFK aliasing in non-nil store check.
* cmake: set cmake_minimum_required only once
* cmake: fix warning about minimum required version
* ci: add a workflow for testing with AVX512 enabled
* test: introduce a helper read_file
* OSX/iOS/ARM64: Fix generation of Mach-O object files.
* OSX/iOS/ARM64: Fix bytecode embedding in Mach-O object file.
* build: introduce LUAJIT_USE_UBSAN option
* ci: enable UBSan for sanitizers testing workflow
* cmake: add the build directory to the .gitignore
* Prevent sanitizer warning in snap_restoredata().
* Avoid negation of signed integers in C that may hold INT*_MIN.
* Show name of NYI bytecode in -jv and -jdump.

Closes tarantool#9924
Closes tarantool#8473

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
Buristan added a commit to Buristan/tarantool that referenced this issue Jul 5, 2024
* ci: checkout integration workflows to release/3.1
* Correct fix for stack check when recording BC_VARG.
* test: remove inline suppressions of _TARANTOOL
* FFI: Fix ffi.alignof() for reference types.
* FFI: Fix sizeof expression in C parser for reference types.
* FFI: Allow ffi.metatype() for typedefs with attributes.
* FFI: Fix ffi.metatype() for non-raw types.
* Maintain chain invariant in DCE.
* build: introduce option LUAJIT_ENABLE_TABLE_BUMP
* ci: add tablebump flavor for exotic builds
* test: allow `jit.parse` to return aborted traces
* Handle all types of errors during trace stitching.
* Use generic trace error for OOM during trace stitching.
* Check for IR_HREF vs. IR_HREFK aliasing in non-nil store check.
* cmake: set cmake_minimum_required only once
* cmake: fix warning about minimum required version
* ci: add a workflow for testing with AVX512 enabled
* test: introduce a helper read_file
* OSX/iOS/ARM64: Fix generation of Mach-O object files.
* OSX/iOS/ARM64: Fix bytecode embedding in Mach-O object file.
* build: introduce LUAJIT_USE_UBSAN option
* ci: enable UBSan for sanitizers testing workflow
* cmake: add the build directory to the .gitignore
* Prevent sanitizer warning in snap_restoredata().
* Avoid negation of signed integers in C that may hold INT*_MIN.
* Show name of NYI bytecode in -jv and -jdump.

Closes tarantool#9924
Closes tarantool#8473

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
Buristan added a commit to Buristan/tarantool that referenced this issue Jul 5, 2024
* Correct fix for stack check when recording BC_VARG.
* test: remove inline suppressions of _TARANTOOL
* FFI: Fix ffi.alignof() for reference types.
* FFI: Fix sizeof expression in C parser for reference types.
* FFI: Allow ffi.metatype() for typedefs with attributes.
* FFI: Fix ffi.metatype() for non-raw types.
* Maintain chain invariant in DCE.
* build: introduce option LUAJIT_ENABLE_TABLE_BUMP
* ci: add tablebump flavor for exotic builds
* test: allow `jit.parse` to return aborted traces
* Handle all types of errors during trace stitching.
* Use generic trace error for OOM during trace stitching.
* Check for IR_HREF vs. IR_HREFK aliasing in non-nil store check.
* cmake: set cmake_minimum_required only once
* cmake: fix warning about minimum required version
* ci: add a workflow for testing with AVX512 enabled
* test: introduce a helper read_file
* OSX/iOS/ARM64: Fix generation of Mach-O object files.
* OSX/iOS/ARM64: Fix bytecode embedding in Mach-O object file.
* build: introduce LUAJIT_USE_UBSAN option
* ci: enable UBSan for sanitizers testing workflow
* cmake: add the build directory to the .gitignore
* Prevent sanitizer warning in snap_restoredata().
* Avoid negation of signed integers in C that may hold INT*_MIN.
* Show name of NYI bytecode in -jv and -jdump.

Closes tarantool#9924
Closes tarantool#8473

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
Buristan added a commit to Buristan/tarantool that referenced this issue Jul 5, 2024
* ci: checkout integration workflows to release/3.1
* Correct fix for stack check when recording BC_VARG.
* test: remove inline suppressions of _TARANTOOL
* FFI: Fix ffi.alignof() for reference types.
* FFI: Fix sizeof expression in C parser for reference types.
* FFI: Allow ffi.metatype() for typedefs with attributes.
* FFI: Fix ffi.metatype() for non-raw types.
* Maintain chain invariant in DCE.
* build: introduce option LUAJIT_ENABLE_TABLE_BUMP
* ci: add tablebump flavor for exotic builds
* test: allow `jit.parse` to return aborted traces
* Handle all types of errors during trace stitching.
* Use generic trace error for OOM during trace stitching.
* Check for IR_HREF vs. IR_HREFK aliasing in non-nil store check.
* cmake: set cmake_minimum_required only once
* cmake: fix warning about minimum required version
* ci: add a workflow for testing with AVX512 enabled
* test: introduce a helper read_file
* OSX/iOS/ARM64: Fix generation of Mach-O object files.
* OSX/iOS/ARM64: Fix bytecode embedding in Mach-O object file.
* build: introduce LUAJIT_USE_UBSAN option
* ci: enable UBSan for sanitizers testing workflow
* cmake: add the build directory to the .gitignore
* Prevent sanitizer warning in snap_restoredata().
* Avoid negation of signed integers in C that may hold INT*_MIN.
* Show name of NYI bytecode in -jv and -jdump.

Closes tarantool#9924
Closes tarantool#8473

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
Buristan added a commit to Buristan/tarantool that referenced this issue Jul 5, 2024
* Correct fix for stack check when recording BC_VARG.
* test: remove inline suppressions of _TARANTOOL
* FFI: Fix ffi.alignof() for reference types.
* FFI: Fix sizeof expression in C parser for reference types.
* FFI: Allow ffi.metatype() for typedefs with attributes.
* FFI: Fix ffi.metatype() for non-raw types.
* Maintain chain invariant in DCE.
* build: introduce option LUAJIT_ENABLE_TABLE_BUMP
* ci: add tablebump flavor for exotic builds
* test: allow `jit.parse` to return aborted traces
* Handle all types of errors during trace stitching.
* Use generic trace error for OOM during trace stitching.
* Check for IR_HREF vs. IR_HREFK aliasing in non-nil store check.
* cmake: set cmake_minimum_required only once
* cmake: fix warning about minimum required version
* ci: add a workflow for testing with AVX512 enabled
* test: introduce a helper read_file
* OSX/iOS/ARM64: Fix generation of Mach-O object files.
* OSX/iOS/ARM64: Fix bytecode embedding in Mach-O object file.
* build: introduce LUAJIT_USE_UBSAN option
* ci: enable UBSan for sanitizers testing workflow
* cmake: add the build directory to the .gitignore
* Prevent sanitizer warning in snap_restoredata().
* Avoid negation of signed integers in C that may hold INT*_MIN.
* Show name of NYI bytecode in -jv and -jdump.

Closes tarantool#9924
Closes tarantool#8473

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
Buristan added a commit to Buristan/tarantool that referenced this issue Jul 5, 2024
* Correct fix for stack check when recording BC_VARG.
* test: remove inline suppressions of _TARANTOOL
* FFI: Fix ffi.alignof() for reference types.
* FFI: Fix sizeof expression in C parser for reference types.
* FFI: Allow ffi.metatype() for typedefs with attributes.
* FFI: Fix ffi.metatype() for non-raw types.
* Maintain chain invariant in DCE.
* build: introduce option LUAJIT_ENABLE_TABLE_BUMP
* ci: add tablebump flavor for exotic builds
* test: allow `jit.parse` to return aborted traces
* Handle all types of errors during trace stitching.
* Use generic trace error for OOM during trace stitching.
* Check for IR_HREF vs. IR_HREFK aliasing in non-nil store check.
* cmake: set cmake_minimum_required only once
* cmake: fix warning about minimum required version
* ci: add a workflow for testing with AVX512 enabled
* test: introduce a helper read_file
* OSX/iOS/ARM64: Fix generation of Mach-O object files.
* OSX/iOS/ARM64: Fix bytecode embedding in Mach-O object file.
* build: introduce LUAJIT_USE_UBSAN option
* ci: enable UBSan for sanitizers testing workflow
* cmake: add the build directory to the .gitignore
* Prevent sanitizer warning in snap_restoredata().
* Avoid negation of signed integers in C that may hold INT*_MIN.
* Show name of NYI bytecode in -jv and -jdump.

Closes tarantool#9924
Closes tarantool#8473

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
Buristan added a commit that referenced this issue Jul 8, 2024
* ci: checkout integration workflows to release/3.1
* Correct fix for stack check when recording BC_VARG.
* test: remove inline suppressions of _TARANTOOL
* FFI: Fix ffi.alignof() for reference types.
* FFI: Fix sizeof expression in C parser for reference types.
* FFI: Allow ffi.metatype() for typedefs with attributes.
* FFI: Fix ffi.metatype() for non-raw types.
* Maintain chain invariant in DCE.
* build: introduce option LUAJIT_ENABLE_TABLE_BUMP
* ci: add tablebump flavor for exotic builds
* test: allow `jit.parse` to return aborted traces
* Handle all types of errors during trace stitching.
* Use generic trace error for OOM during trace stitching.
* Check for IR_HREF vs. IR_HREFK aliasing in non-nil store check.
* cmake: set cmake_minimum_required only once
* cmake: fix warning about minimum required version
* ci: add a workflow for testing with AVX512 enabled
* test: introduce a helper read_file
* OSX/iOS/ARM64: Fix generation of Mach-O object files.
* OSX/iOS/ARM64: Fix bytecode embedding in Mach-O object file.
* build: introduce LUAJIT_USE_UBSAN option
* ci: enable UBSan for sanitizers testing workflow
* cmake: add the build directory to the .gitignore
* Prevent sanitizer warning in snap_restoredata().
* Avoid negation of signed integers in C that may hold INT*_MIN.
* Show name of NYI bytecode in -jv and -jdump.

Closes #9924
Closes #8473

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
Buristan added a commit that referenced this issue Jul 8, 2024
* Correct fix for stack check when recording BC_VARG.
* test: remove inline suppressions of _TARANTOOL
* FFI: Fix ffi.alignof() for reference types.
* FFI: Fix sizeof expression in C parser for reference types.
* FFI: All
64ED
ow ffi.metatype() for typedefs with attributes.
* FFI: Fix ffi.metatype() for non-raw types.
* Maintain chain invariant in DCE.
* build: introduce option LUAJIT_ENABLE_TABLE_BUMP
* ci: add tablebump flavor for exotic builds
* test: allow `jit.parse` to return aborted traces
* Handle all types of errors during trace stitching.
* Use generic trace error for OOM during trace stitching.
* Check for IR_HREF vs. IR_HREFK aliasing in non-nil store check.
* cmake: set cmake_minimum_required only once
* cmake: fix warning about minimum required version
* ci: add a workflow for testing with AVX512 enabled
* test: introduce a helper read_file
* OSX/iOS/ARM64: Fix generation of Mach-O object files.
* OSX/iOS/ARM64: Fix bytecode embedding in Mach-O object file.
* build: introduce LUAJIT_USE_UBSAN option
* ci: enable UBSan for sanitizers testing workflow
* cmake: add the build directory to the .gitignore
* Prevent sanitizer warning in snap_restoredata().
* Avoid negation of signed integers in C that may hold INT*_MIN.
* Show name of NYI bytecode in -jv and -jdump.

Closes #9924
Closes #8473

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants
0