Fixed a bug where an index scan would be used to solve an OPTIONAL MATCH incorrectly.

Fix bug where using aggregation in a reduce expression could lead to the wrong result.

Fixes a bug where the order of results in a query with an ORDER BY could be wrong, if the query plan included a NodeLeftOuterHashJoin.

Fix bug with exists on nullable node or relationship property

Fixed a bug where count store lookup would be incorrectly planned for a pattern with a multi-labeled node.

Fix error Index cursor cannot have transaction state with values and without values simultaneously happening for some queries that used the index when there are updates in the transaction state.

Fixes various bugs where the Eager operator was not being correctly planned.

• Eager after an OR of label predicates was missing, e.g. in ... MATCH (b:B) WHERE b:B2 OR b:B3 SET a:B2 ...
• Eager after an OPTIONAL MATCH was missing, e.g. in ...OPTIONAL MATCH (b:B) MATCH (c) SET c:B ...
• Eager after any MATCH clause with a single starting point that does not solve the conflicting predicate was missing, e.g. in ... MATCH (n:N:M), (x) SET x:M ... when starting with a NodeByLabelScan on n:N.
• Eager after shortestPath was missing, e.g. in ... MATCH p = shortestPath((a:A)-[:T*]-(b:B)), (c) CREATE (a)-[:T]->(c) ...

Don't overflow on --9223372036854775808. Github issue: neo4j/neo4j#12726

Fixed a bug where running an aggregating query through Fabric would fail with namespacing error.

Importer provides correct estimates for compressed csv files

Fixes a log pruning comparison issue in volumetric check point threshold

Prevent crash in dbms.killTransaction when log level is set to debug

Fixes an issue decrementing node degrees where deleting a relationship from a node could sometimes, if the node had self-relationships, decrement the degree twice

Special handling of unrelated SSL policies in the backup command

Fixes an issue where an unexpected concurrent closing of a page cursor which was in the middle of doing page faulting might leave a latch unreleased, trapping all future cursors that wanted to look at that page.

Update dependencies due to security issues

Jetty 9.4.17 -> 9.4.34 - https://nvd.nist.gov/vuln/detail/CVE-2020-27216 Jersey 2.29 -> 2.32 - SRCCLR-SID-3606 Jackson 2.10.0 -> 2.10.5 - For version compatibility

Upgrade the dependencies of SLF4j and Shiro to latest stable versions in order to mitigate security vulnerabilities.

Fix issue where queries like WITH *, count(a) AS c ORDER BY c RETURN a were failing with no variable a in scope.

Fixed a bug where aggregation followed by a map projection would cause a query to hang or fail with There was a cycle in names error.

Fixes a scoping issue where nested pattern comprehensions were evaluated in the wrong scope leading to wrong results. An example of such a query:

MATCH (f:F)<-[:A]-(p)
WHERE
  ALL(condition IN [
    (p)-[:B]->(d:D)
    d.name CONTAINS 'foo'
  ] WHERE condition)
RETURN p { .name }

Fix bug where predicate in WHERE clause was ignored for queries combining

variable lenght pattern and DISTINCT.

Fix error where we could get an error Missing kernel to cypher mapping for index behavior: EVENTUALLY_CONSISTENT in Cypher.

Fixed an issue where dbms.listTransactions and dbms.listQueries could fail if the server starts up with a database in the stopped state.

Fixed bug where clusters could get stuck bootstrapping in the event of network issues or instance restarts.

• Prevoiusly there was a small window for a race where a member would mistakenly believe it had failed to publish a raft Id. It would then observe the raft id it published and, assuming someone else had bootstrapped, wait forever.
• Now, when we observe a raftId while trying to bind we check whether someone else actually published it before we wait.
• In the even that we published it but believe we failed, we simply retry publishing. It is an idempotent operation.

• Added RaftBinderTest which tests whether a server will eventually try to bootstrap if it succeeds in publishing a raftId but believes that it has failed.
• Add didBootstrapDatabase check
• Added RaftId->MemberId map to BootstrappedRaftsMessage
• Added bootstrappers to bootstrap state and tested
• Exposed didBootstrap lookup through CoreTopologyService

Make failed indexes restartable again

Fix bug where indexes and constraint might be incorrectly migrated when upgrading from standard to high_limit format.

Fixes an ID allocation issue when importing a large amount of relationships so that double-record-unit allocation got enabled. This could result in some relationships getting one of their record units overwritten

Don't fail SslPolicy creation if there are dotfiles present in the certificate directories

Fix bug with potential wrong order for queries with "ORDER BY"

that use an index in the presence of concurrent inserts into that index.

Improve error message when index key size limit is violated during index population, include entity id index description.

Fix a bug in the code generation for pipelined runtime, that under certain rare plans could cause that not all the necessary variables in a row were copied between pipelines, leading to null pointer exceptions or unexpected results.

Fix bugs in planning (NOT) EXISTS subqueries within different horizons

Fix issue where an error was thrown during planning for some complex COLLECT/UNWIND queries.

Fix bug when combining * and a map projection in the same projection, e.g. WITH , s {.}...

Update Apache Shiro to 1.5.3

4.0 planner cardinality fix for OPTIONAL MATCH under APPLY

Fix bug which could end up in Can't coerce NO_VALUE to Path errors being thrown

when using user-defined functions.

Fixed a bug where using node:Label check on a node returned from a procedure would result in a planner error.

When running Cypher queries with PROFILE, the amount of rows for operator would be wrong in certain cases if the pipelined runtime is used to execute a query. This has been fixed.

Retain solved hints from the right-hand side of a union.

Prevent excess memory usage in OrLeafPlanner with many ORs

Accept dots in database names

Mdx guide support

Fix reading server configuration with the PUBLIC user role

Update the UI color schemes (dark, light, contrast)

Example:

Vertical align result table cell values at row top

Upgrade neo4j-driver to 4.1.0

Fix integer rendering in table recursively + JSON export

Make joinSolverStep generate joins when goal is compacted (#5620)

Fix regression due to resource tracking of resource managers

Fixes a bug with the error message Did not find any type information for expression .... An example query that would trigger this bug is

MATCH (user:User)
WHERE ($range[0] > user.id OR user.id > $range[1])
RETURN user
LIMIT 1

Fixes a bug with the error message Tried to invalidate a cached property ... but no slot was found for the entity name ... that could appear in queries that set or remove properties.

Fix bug in cardinality calculation where the presence of unrelated arguments could cause the planner to pick a LabelScan instead of an Argument operator.

Fix a bug where users were seing Expected a string value for ``toLower``, but got: NO_VALUE; consider converting it to a string with toString(). unexpectedly.

Fix show current users privileges for external user

Fix bug where existential subqueries inside OR predicates were not properly supported.

Fixes $12433, when combining OPTIONAL MATCH, aggregation and map-projection, in some scenarios we

failed with: Can't coerce NO_VALUE to Map.

Adds a procedure db.ping() :: (success :: BOOLEAN?).

This procedure can be used by client side tooling to test whether they are correctly connected to a database. The procedure is available in all databases and always returns true. A faulty connection can be detected by not being able to call this procedure.

Fix bug with exists and dynamic map lookup

Fixes incorrect ordering for queries with "ORDER BY" that got plans with NodeHashJoin right of Apply. The bug only existed in the pipelined runtime.

Fix bug with PatternComprehension and "ORDER BY".

Combination of PatternComprehension, MapProjection and "ORDER BY" could sometimes lead to SteackOverflow exceptions.

Fix bug with VarExpand(Pruning) failing to compile with Tried overwriting already taken variable name in slotted and pipelined runtimes. Solve issue $12441.

Fixed a bug that would result in org.neo4j.driver.exceptions.DatabaseException: org.neo4j.cypher.internal.physicalplanning.ast.NullCheckReference.

Fixed a bug with namespacing of existential subqueries using an ambiguous outer variable.

Update shiro to 1.5.2 to fix BlackDuck warning

Cypher-shell version 4.0.3

Fixes a bug where the security graph wasn't initialized unless native was used in either dbms.security.authentication_providers or dbms.security.authorization_providers when running in a clustered environment

Cypher-shell version 4.0.4

Introducing a new global metric (same level as Bolt Server, JVM and Netty Server metrics) about the number of database operations in the instance. Six counters are exposed:

• Number of database creations: neo4j.db.operation.count.create.
• Number of database starts: neo4j.db.operation.count.start.
• Number of database stops: neo4j.db.operation.count.stop.
• Number of database drops: neo4j.db.operation.count.drop.
• Number of failed operations: neo4j.db.operation.count.failed. For each database it counts only once. So if more operation (same or different) fail on the same database this counter will increase only once.
• Number of recovered operations: neo4j.db.operation.count.recovered. This counter increases once the database leaves the failed state.

From the numbers above it is possible to calculate, how many databases are started, stopped or in a failed state on a DBMS instance.

The database could previously leave the "temp-bootstrap" directory around as part of cluster formation and especially as part of a series of failed formation attempts this lead to them getting stacked further and further on top of each other (i.e. temp-bootstrap/temp-bootstrap/temp-bootstrap/...). This should only have affected the bootstrapping of the system database. For installations which are functional but have this directory laying around unnecessarily it will simply be removed now for any database.

Parsing or compilation related errors are now also logged in the query log.

Fixes a bug where you could not revoke a sub-privilege that was denied when having the compound privilege granted.

E.g. DENY NAME MANAGEMENT ON DATABASE * TO admin followed by REVOKE DENY NAME MANAGEMENT ON DATABASE * FROM admin

Revert the deprecation of the config settings dbms.security.ldap.authorization.group_to_role_mapping, dbms.security.procedures.default_allowed and dbms.security.procedures.roles, since replacements are not in place yet.

Database and graph names are now case-insensitive for all security administration commands.

Fixed bug where getDegree would leak blocked relationships

This call was used in Cypher to answer queries like RETURN size((a)-->()). If the relationships or the end node labels were not visible, either due to blacklisting (DENY) or lack of whitelisting (GRANT), the correct answer would only be returned for non-supernodes. With supernodes, the optimization would not consider security and give too high counts.

The fix applies to both the Cypher surface and the Core API (NodeEntity).

The neo4j-admin copy command now correctly ignores indexes and constraints that cannot be correctly read from the schema store.

The neo4j-admin load command now has an --info option that will make the command print information about the archive, instead of loading it.

Pass through exception message in transaction event handlers

Fix a problem where the combination of parallel apoc.periodic.iterate, and dbms.logs.query.parameter_logging_enabled=true, could lead to a JVM crash.

Last line in the configuration file is no longer ignored if there is a missing new line

Fix dbms.logs.gc.rotation.size validation error

Fix bug where store upgrade could see IndexOutOfBoundsException when upgrading store that was much larger than available memory.

Add jvm argument, -Dorg.neo4j.kernel.impl.storemigration.RecordStorageMigrator.migration_threads=, to set number of threads used to read and write when upgrading store. By setting a lower value the contention on disk can be reduced. Default value is the number of processors available to the process.

db.info and dbms.info functions to retrieve database and dbms identity information.

Fix a bug that could cause a class cast exception when accessing a cached property on a node value that was null.

Fix issue that could lead to errors when using map projections with projections that evaluates to null.

$fixes: 12398

Fixed a bug where outer hash join would be incorrectly planned to solve OPTIONAL MATCH with non-join dependencies.

Track memory used by Expand(Into)

The currently logged-in user, even if not an admin, can use SHOW USER username PRIVILEGES to lookup its own privileges

Track memory in pruning-var-expand

Fix bug where the wrong results could be returned in pipelined

runtime whenever a filter came after a distinct.

Fixed bug in compiled expressions where a null path from an OPTIONAL MATCH inserted into nested functions lead to an unexpected error.

Fixed a bug where shadowed variable from outer scope would be accessible inside list comprehension when combined with aggregating function.

Fix so that SHOW DATABASES only show databases that the current user has access privilege on.

Fixed an IDP planner edge case where complex joins fail to find a solution due to the presence of ignored argument variables

Fixes regressions with PatternComprehensions inside of head or single element accesses/list slices . Expamples head([ (a)-->(b) | b.prop ]), [ (a)-->(b) | b.prop ][1], [ (a)-->(b) | b.prop ][..5]

Improve error message for procedure calls

Added a nicer error message if the planner fails to find a plan

Add support for sub-type values in procedure parameter default values

Fix bug where Number-typed procedure parameters did not support default values

Create index and create constraint are no longer idempotent

Fix a planning problem where the planner wasn't using the index after nodes have been deleted.

Combining index read, DISTINCT, and MERGE could in some

situations end up with an error saying Uh oh... There was no slot for...

Solved a bug in compiled runtime where complex NodeHashJoins sometimes led to wrong result.

Fixes bug in slotted runtime, when a property is read from index and then a node property of the same name is updated later in the query.

E.g. MATCH (n:Label) WHERE n.prop > 0 WITH collect(n.prop) AS ns MATCH (n:Label) SET n.prop = 0

Enhance split function to take multiple delimiters

Adding ProcedureCallContext to Context that can be used by procedures. This will give procedures information about the context in which they were called, specifically which return items were requested via yield. Based on this, the procedure can safely decide to skip calculating values that are not yielded anyway. The procedures db.labels and db.relationshipTypes have taken advantage of this for performance optimizations.

Solved a bug in slotted runtime where complex NodeHashJoins sometimes led to wrong result.

Bound calculated selectivity of range seek to exact match selectivity...

Allow queries like OPTIONAL MATCH (a) MATCH (a)-->(b) without a WITH

between the clauses.

Adding ProcedureCallContext to Context that can be used by procedures. This will give procedures information about the context in which they were called, specifically which return items were requested via yield. Based on this, the procedure can safely decide to skip calculating values that are not yielded anyway. The procedures db.labels and db.relationshipTypes have taken advantage of this for performance optimizations.

Cypher now supports executing sub-queries. This enables, among other things, processing results of a union query, which was not possible before. The syntax for this looks like that:

CALL {
  MATCH (x:X)
  RETURN x
  UNION
  MATCH(x:Y)
  RETURN x
}
RETURN x ORDER BY x.foo

Upgrade Apache Shiro to 1.4.1 to mitigate DoS vulnerability

Post-planning property read pushdown

Queries with variables defined before optional matches that contain OR predicates and can be planned as UNION queries were failing to retain the incoming variables.

Allows queries like MATCH (n) WHERE id(n)=20.0 RETURN n to return the node with the id 20.

Procedures can now get information from ProcedureCallContext about the name of the database it is running on and if its the system database

Spatial distance queries will now be solved by a NodeUniqueIndexSeekByRange instead of a NodeByLabelScan when an uniqueness constraint exists.

Cache property reads in the row, if the same property is accessed more than once in the query. For example the query MATCH (n:N) WHERE n.prop > 10 RETURN n.prop would only have cached n.prop previously, if it could get the value from an index. Now, it will be cached in any case. This adds caching of relationship properties as well.

The plan description is changed such that both cache reads and writes appear as cache[n.prop] in the Other column.

Fix planning bug with pattern-predicates on var-length-expand patterns, which would get overly expensive plans including RollUpApplies. This bug is only present in 3.5.6.

Rarely the planner was not able to find a join between two perfectly joinable plans which would then result in a "Found no solution for block with size 0" error. This is fixed now.

Update from Scala 2.11 to Scala 2.12

New physical Operator "Ordered Distinct" that performs a distinct operation when the input is sorted.

Adding existential subqueries to Cypher, on the form:

MATCH (n)
WHERE EXISTS {
  MATCH (n)-[r]->(p)
  WHERE p.prop > 2
}
RETURN n

Fixes problem in procedure dbms.setTXMetaData() where null value in the given map could lead to a null pointer exception

Fixed distance queries that used < and ranged over the dateline and would miss points exactly on the dateline

fix scala.MatchError crash on DELETE last(col)

The Cypher planner now can use index provided properties (instead of going to the property store) in the following case: MATCH (n:Label) RETURN n.prop given an index and an existence or node key constraint on :Label(prop).

Add procedure for replanning

Show file name and line number for LOAD CSV errors

The Cypher planner now can use index backed order by for the min and max functions with a range predicate and thus avoid sorting/aggregation for queries like: MATCH (n:Label) WHERE n.prop > 0 RETURN min(n.prop) given an index on :Label(prop).

Fix issue where queries with many disjunctive index seeks (... n.age = 3 OR n.name = 'George' ...) would plan forever.

Fix issue with compiled runtime and count(), where transient entity inconsistencies would make some aggregating queries (e.g. MATCH (n:Artist) WHERE n.name = 'Madonna' RETURN count(n)) return fewer rows than expected.

Fixes an issue with having a DISTINCT (that is on the node instead of the property) between an index read and the usage of the property.

E.g MATCH (n:Label) WHERE n.prop = 42 WITH DISTINCT n RETURN n.prop

Fix an error that made the timestamp function case sensitive, https://github.com/neo4j/neo4j/issues/12097

The Cypher planner now can use index backed property values for aggregation functions. That implies that queries like: MATCH (n:Label) WITH n.prop AS property RETURN sum(property) can be solved with a NodeIndexScan rather than a NodeByLabelScan (given an index on :Label(prop)).

Make system-graph the built-in default auth provider

Obsfucate LDAP system password in debug log

When ordering both by aliased (var01) and unaliased (p.name)

columns, we would possibly swap them. This is now fixed.

Updating after undirected match with predicate

Fix issue with PROFILE and EXPLAIN of queries with unusual variable names