perf: explore pushing parts of MVCC or KV evaluation into C++ #17172
Comments
|
Go has two ways to interface with foreign code: cgo and assembly. The assembly interface doesn't interact with the scheduler, so it's much faster (but it has limitations such as not being suitable for code that may make system calls). There are tricks like https://github.com/minio/c2goasm for calling C code via the assembly interface; we may be able to do something like this to reduce the cost of crossing the boundary for frequently-called functions. |
|
|
|
I prototyped a tool which uses reflection to generate C++ to read a Go struct. It can't handle maps, channels or interfaces, but everything else is fairly straightforward. Another possibility for using this tool would be to move either the timestamp cache or raft entry cache into C++. Both data structures have interfaces that would be awkward to shoehorn into the cgo interface without overhead. |
|
Also, see golang/go#16051 for how the Go race detector calls into C++ without the cgo overhead. This is low-level and architecture specific, but might be worthwhile. |
|
just curious, how much confident are you that the GC doesn't rewrite pointers / move data structures concurrently? (e.g. to reduce fragmentation) |
I'm 100% confident that it doesn't. The concern is whether a future version of the GC will do so. The current cgo pointer passing rules can be seen as leaving open this possibility, though I'm not aware of any work that is pushing towards it. |
Generate C++ code for read-only access of a Go structure. See cockroachdb#17172
Generate C++ code for read-only access of a Go structure. See cockroachdb#17172
|
With the introduction of the arena skiplist timestamp-cache and reimplementing |
We've observed in the past that minimizing Go/C++ crossings in
storage/engine/mvcc.gois a useful performance optimization. But many of the interesting methods inmvcc.gostill perform multiple Cgo calls. And stepping up a level, evaluation of a KV operation can often perform multiple MVCC operations.The challenge in pushing more logic into C++ is in the Go/C++ boundary. Currently, Go code manually fills in C (not C++) structures and passes those C structures to C routines. When data is returned from C++ the Go code manually copies it from the C structs. So far, we've avoided passing any complicated structure and thus the C++ interface is very basic, knowing only about strings and timestamps.
Cgo does not provide a facility for passing Go structures to C. I think we can provide that facility ourselves. Specifically, we can use
reflectto auto-generate C++ code that can access a Go data structure. Usingreflectwe can find the types and byte offsets of any field for a given Go type. Auto-generating C++ code from these byte offsets is then relatively straightforward.Note that passing an arbitrary Go structure to C++ would violate the rules in https://golang.org/cmd/cgo/#hdr-Passing_pointers. Those rules are somewhat stricter than necessary and we'd definitely be playing on the edge here. The auto-generated code could only be used for reading the Go structure, not for writing as we'd definitely not want to deal with supporting the GC write barrier. This implies that returning a structure from C++ to Go would be more limited. A final limitation is that this approach would not allow C++ to iterate over a Go map, but so far none of the protos in
roachpbuse Go maps.The proposed steps of exploration are:
MVCCGet) to C++ and benchmark the performance improvement.The text was updated successfully, but these errors were encountered: