8000 vinyl: le/lt iterator crashes in vy_mem_iterator_restore · Issue #2614 · tarantool/tarantool · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

vinyl: le/lt iterator crashes in vy_mem_iterator_restore #2614

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
locker opened this issue Jul 18, 2017 · 3 comments
Closed

vinyl: le/lt iterator crashes in vy_mem_iterator_restore #2614

locker opened this issue Jul 18, 2017 · 3 comments
Assignees
@Gerold103
Labels
bug Something isn't working vinyl
Milestone
1.7.5

Comments

@locker
Copy link
Member
locker commented Jul 18, 2017

Branch 1.7 commit 4cf7601

#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007f58857a13fa in __GI_abort () at abort.c:89
#2  0x00007f5885798e37 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x559b3cbe1888 "vy_stmt_compare(itr->curr_stmt, last_stmt, def) <= 0",
    file=file@entry=0x559b3cbe1238 "/home/vlad/src/tarantool/src/box/vy_mem.c", line=line@entry=856, function=function@entry=0x559b3cbe2180 <__PRETTY_FUNCTION__.8958> "vy_mem_iterator_restore") at assert.c:92
#3  0x00007f5885798ee2 in __GI___assert_fail (assertion=0x559b3cbe1888 "vy_stmt_compare(itr->curr_stmt, last_stmt, def) <= 0", file=0x559b3cbe1238 "/home/vlad/src/tarantool/src/box/vy_mem.c", line=856,
    function=0x559b3cbe2180 <__PRETTY_FUNCTION__.8958> "vy_mem_iterator_restore") at assert.c:101
#4  0x0000559b3c923e74 in vy_mem_iterator_restore (vitr=0x559b3d192cf0, last_stmt=0x559b3d190e60, ret=0x559b3d192d88, stop=0x7f588112fb12) at /home/vlad/src/tarantool/src/box/vy_mem.c:856
#5  0x0000559b3c911be2 in vy_merge_iterator_next_key (itr=0x7f5881130380, ret=0x7f588112fbb8) at /home/vlad/src/tarantool/src/box/vinyl.c:4459
#6  0x0000559b3c912c4e in vy_read_iterator_merge_next_key (itr=0x7f5881130328, ret=0x7f588112fbb8) at /home/vlad/src/tarantool/src/box/vinyl.c:5317
#7  0x0000559b3c912fb8 in vy_read_iterator_next (itr=0x7f5881130328, result=0x7f588112fc28) at /home/vlad/src/tarantool/src/box/vinyl.c:5429
#8  0x0000559b3c915613 in vy_cursor_next (env=0x559b3d174d90, c=0x7f5881130230, result=0x7f588112fc78) at /home/vlad/src/tarantool/src/box/vinyl.c:6428
#9  0x0000559b3c901f79 in iterator_next (base_it=0x559b3d192b40) at /home/vlad/src/tarantool/src/box/vinyl_index.cc:164
#10 0x0000559b3c8f44f2 in Handler::executeSelect (this=0x559b3d17c010, space=0x559b3d1a17a0, index_id=0, iterator=4, offset=0, limit=10, key=0x7f588110c039 "\315\034:", 'P' <repeats 197 times>..., port=0x7f588112fdf0)
    at /home/vlad/src/tarantool/src/box/engine.cc:203
#11 0x0000559b3c96baf3 in box_select (port=0x7f588112fdf0, space_id=512, index_id=0, iterator=4, offset=0, limit=10, key=0x7f588110c038 "\221\315\034:", 'P' <repeats 196 times>..., key_end=0x7f588110c03c 'P' <repeats 200 times>...)
    at /home/vlad/src/tarantool/src/box/box.cc:752
#12 0x0000559b3c9901a0 in lbox_select (L=0x412094a0) at /home/vlad/src/tarantool/src/box/lua/misc.cc:94
#13 0x0000559b3c9e233b in lj_BC_FUNCC ()
#14 0x0000559b3ca046e7 in lua_pcall (L=0x412094a0, nargs=0, nresults=0, errfunc=0) at lj_api.c:1129
#15 0x0000559b3c9bac0a in luaT_call (L=0x412094a0, nargs=0, nreturns=0) at /home/vlad/src/tarantool/src/lua/utils.c:931
#16 0x0000559b3c9b4a4c in lua_fiber_run_f (ap=0x7f58810006e8) at /home/vlad/src/tarantool/src/lua/fiber.c:267
#17 0x0000559b3c8b9a94 in fiber_cxx_invoke(fiber_func, typedef __va_list_tag __va_list_tag *) (f=0x559b3c9b498c <lua_fiber_run_f>, ap=0x7f58810006e8) at /home/vlad/src/tarantool/src/fiber.h:647
#18 0x0000559b3c9c9e17 in fiber_loop (data=0x0) at /home/vlad/src/tarantool/src/fiber.c:641
#19 0x0000559b3cb9cc8f in coro_init () at /home/vlad/src/tarantool/third_party/coro/coro.c:110

Reproducer:

fiber = require 'fiber'

box.cfg{}

count = 10000

s = box.schema.space.create("test", {engine='vinyl'})
s:create_index('pk')

fiber.create(function()
    while true do
        s:select(math.random(count), {iterator = box.index.LE, limit = 10})
        fiber.sleep(0.01)
    end
end)

fiber.create(function()
    while true do
        box.snapshot()
        fiber.sleep(0.01)
    end
end)

for i = 1, count do
    s:replace{math.random(count)}
end

os.exit(0)
@locker locker added bug Something isn't working vinyl labels Jul 18, 2017
@locker locker added this to the 1.7.5 milestone Jul 18, 2017
alyapunov added a commit that referenced this issue Jul 21, 2017
@alyapunov
vinyl: fix a crash in mem_iterator_restore
a417be2 
Fix a bug in restoration LE/LT when last_stmt is not in the mem
(it came from another mem or run).

Fix a bug in restoration after EOF reached.

Add a unit test of mem_iterator_restore.

Fixes #2614
rtsisyk pushed a commit that referenced this issue Jul 25, 2017
@alyapunov @rtsisyk
vinyl: fix a crash in mem_iterator_restore
e576ca9 
Fix a bug in restoration LE/LT when last_stmt is not in the mem
(it came from another mem or run).

Fix a bug in restoration after EOF reached.

Add a unit test of mem_iterator_restore.

Fixes #2614
@rtsisyk
Copy link
Contributor
rtsisyk commented Jul 26, 2017

This problem has been partially fixed. However, setting count = 10000000 in the example above still crashes Vinyl :(

@rtsisyk rtsisyk assigned Gerold103 and unassigned alyapunov Jul 26, 2017
@rtsisyk rtsisyk added the prio1 label Jul 26, 2017
@Gerold103
Copy link
Collaborator
Gerold103 commented Jul 27, 2017 •
edited
Loading 
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
  * frame #0: 0x00007fffabeb3d42 libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fffabfa1457 libsystem_pthread.dylib`pthread_kill + 90
    frame #2: 0x00007fffabe19420 libsystem_c.dylib`abort + 129
    frame #3: 0x00007fffabde0893 libsystem_c.dylib`__assert_rtn + 320
    frame #4: 0x00000001000a24fc tarantool`vy_cache_add(cache=0x0000000000503cd8, stmt=0x0000000000321af0, prev_stmt=0x0000000000321ac0, key=0x000000001ae150d0, order=589505315) at vy_cache.c:329
    frame #5: 0x000000010009e3eb tarantool`vy_read_iterator_next(itr=0x0000000002130328, result=0x000000000212fad8) at vy_read_iterator.c:1399
    frame #6: 0x0000000100061756 tarantool`vy_cursor_next(env=0x0000000000804200, c=0x0000000002130230, result=0x000000000212fb18) at vinyl.c:4930
    frame #7: 0x00000001000578e9 tarantool`iterator_next(base_it=0x000000001ae08840) at vinyl_index.cc:164
    frame #8: 0x0000000100044d80 tarantool`Handler::executeSelect(this=0x0000000000503460, (null)=0x0000000000000000, space=0x00000000005038c0, index_id=0, iterator=4, offset=0, limit=10, key="?, (null)="PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP"..., port=0x000000000212fd70) at engine.cc:203
    frame #9: 0x00000001000d790b tarantool`::box_select(port=0x000000000212fd70, space_id=512, index_id=0, iterator=4, offset=0, limit=10, key="\x91?, key_end="PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP"...) at box.cc:775
    frame #10: 0x0000000100103ba1 tarantool`lbox_select(L=0x0000000000784068) at misc.cc:94
    frame #11: 0x00000001001535eb tarantool`lj_BC_FUNCC + 52
    frame #12: 0x000000010017eea3 tarantool`lua_pcall(L=0x0000000000784068, nargs=0, nresults=0, errfunc=0) at lj_api.c:1129
    frame #13: 0x0000000100118643 tarantool`luaT_call(L=0x0000000000784068, nargs=0, nreturns=0) at utils.c:931
    frame #14: 0x0000000100111d55 tarantool`lua_fiber_run_f(ap=0x0000000002000670) at fiber.c:264
    frame #15: 0x0000000100004521 tarantool`fiber_cxx_invoke(f=(tarantool`lua_fiber_run_f at fiber.c:259), ap=0x0000000002000670)(__va_list_tag*), __va_list_tag*) at fiber.h:644
    frame #16: 0x000000010012897b tarantool`fiber_loop(data=0x0000000000000000) at fiber.c:626
    frame #17: 0x00000001002ba0d7 tarantool`coro_init at coro.c:110

Gerold103 added a commit that referenced this issue Jul 28, 2017
@Gerold103
Introduce vy_cache unit test
d79bf51 
Needed for #2614
Gerold103 added a commit that referenced this issue Jul 31, 2017
@Gerold103
Introduce vy_cache unit test
805a379 
Needed for #2614
Gerold103 added a commit that referenced this issue Aug 1, 2017
@Gerold103
Introduce vy_cache unit test
494f312 
And fix a bug in vy_cache_iterator_restore, which could ignore
restore to a previous statement.

Needed for #2614
Gerold103 added a commit that referenced this issue Aug 1, 2017
@Gerold103
vinyl: fix run_iterator to correctly process LE selects
34972b8 
If the iterator type is ITER_LE and a run_iterator can not find
a start page (all page min_keys are >= select key), it would
iterate over all pages starting from the last.

Fix it by setting search_ended flag at the page search phase, before
calling next_key.

Closes #2614
Gerold103 added a commit that referenced this issue Aug 1, 2017
@Gerold103
vinyl: fix run_iterator to correctly process LE selects
613e73a 
If the iterator type is ITER_LE and a run_iterator can not find
a start page (all page min_keys are >= select key), it would
iterate over all pages starting from the last.

Fix it by setting search_ended flag at the page search phase, before
calling next_key.

Closes #2614
rtsisyk pushed a commit that referenced this issue Aug 2, 2017
@Gerold103 @rtsisyk
Introduce vy_cache unit test
0334e3e 
And fix a bug in vy_cache_iterator_restore, which could ignore
restore to a previous statement.

Needed for #2614
rtsisyk pushed a commit that referenced this issue Aug 2, 2017
@Gerold103 @rtsisyk
vinyl: fix run_iterator to correctly process LE selects
ec8d56c 
If the iterator type is ITER_LE and a run_iterator can not find
a start page (all page min_keys are >= select key), it would
iterate over all pages starting from the last.

Fix it by setting search_ended flag at the page search phase, before
calling next_key.

Closes #2614
@rtsisyk
Copy link
Contributor
rtsisyk commented Aug 2, 2017

1.7.4-372-gec8d56c58

@rtsisyk rtsisyk closed this as completed Aug 2, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Gerold103 Gerold103

Labels
bug Something isn't working vinyl
Projects
None yet
Milestone
1.7.5
Development

No branches or pull requests
315C
4 participants
@locker @rtsisyk @Gerold103 @alyapunov
0