Add a result iterator to all cursor classes #1081
Conversation
90b906f to
5b7b9aa
Compare
295e496 to
0c07d43
Compare
|
@dlax I would like to hear your opinion about a topic related to this improvement. As you surely remember, initially, in psycopg 3.1 the pipeline mode was accumulating all the results in the cursor, but we changed this behaviour in #604 (we warned against using the feature in 3.1.10 and changed the behaviour in 3.2). However the lack of reliability might have been related to other errors that have been since fixed (like #685, fixed in 3.1.18). I see some utility in receiving more than one result, especially considering that - I seem to remember - in pipeline mode you can't use more than one query in the same statement. People could use the pipeline mode to request a batch of objects from different tables and then receive all the result to aggregate on the client: with cur.pipeline():
cur.execute("select * from parent where id = %s", [rec_id])
cur.execute("select * from child where parent_id = %s", [rec_id])
parent = cur.fetchone()
cur.nextset()
children = cur.fetchall()So I was thinking to enable this behaviour on request: what do you think about a parameter on the I haven't reviewed completely the conversations in the tickets linked. Do you have in mind any reasons why this couldn't obviously work? |
| await cur.execute(";".join(["select 1"] * count)) | ||
| assert await alist(cur.results()) == [cur] * count |
There was a problem hiding this comment.
I would test if the result changed too.
| await cur.execute(";".join(["select 1"] * count)) | |
| assert await alist(cur.results()) == [cur] * count | |
| await cur.execute(";".join(["select %s" % i for i in range(count)]) | |
| ress = await alist(res.fetchall() for res in cur.results()) | |
| assert ress == [[(i,)] for i in count] |
| await cur.executemany("select 1", [()] * count, returning=returning) | ||
| assert await alist(cur.results()) == [cur] * returning * count |
There was a problem hiding this comment.
Same here. We could use rowcount to check that even in the returning=False case we iterate on the results. Untested:
| await cur.executemany("select 1", [()] * count, returning=returning) | |
| assert await alist(cur.results()) == [cur] * returning * count | |
| await cur.executemany( | |
| "select * from generate_series(0, %s)", | |
| [(i,) for i in range(count)], | |
| returning=returning, | |
| ) | |
| if returning: | |
| ress = await alist(res.fetchall() for res in cur.results()) | |
| assert ress == [[(j,) for j in range(i)] for i in range(2+1)] | |
| else: | |
| ress = await alist(res.rowcount for res in cur.results()) | |
| assert ress == list(range(count)) |
|
|
||
|
|
||
| @pytest.mark.parametrize("count", [1]) | ||
| def test_results_after_execute(conn, count): |
There was a problem hiding this comment.
I am not sure this method makes sense on server cursors. I would raise a NotSupportedError.
|
|
||
| @pytest.mark.parametrize("count", [1]) | ||
| async def test_results_after_execute(aconn, count): | ||
| async with aconn.cursor() as cur: |
There was a problem hiding this comment.
This is not a server cursor, it is a client cursor. You have to give it a name 🙂
|
|
||
| .. warning:: | ||
|
|
||
| More explicitly, `!fetchall()` alone will not return all the | ||
| values returned! You must iterate on the results using | ||
| `!nextset()`. | ||
| `!results()`. |
There was a problem hiding this comment.
Looking at the docs, nextset() is also used in the from_pg2 page.
Do we want to change that example? Like:
using the usual `!fetch*()` methods. In order to access all the results,
you can use the `~Cursor.results()` method::
>>> cur_pg3.execute("SELECT 1; SELECT 2")
>>> for _ in cur_pg3:
... print(cur_pg3.fetchone())
(1,)
(2,)
I don't feel the need to keep the nextset() example, in all its clunkiness. If anything we could add a paragraph like:
Before Psycopg 3.3 you can use `~Cursor.nextset()` instead.
Closes #1080.