8000 Primitive integers by maximedenes · Pull Request #6914 · rocq-prover/rocq · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Primitive integers #6914

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 4, 2019
Merged

Conversation

maximedenes
Copy link
Member
@maximedenes maximedenes commented Mar 5, 2018

This patch adds support for primitive integers. It can be seen as an evolution of the existing "retroknowledge" machinery, extending the benefits of compact representations of integers to the entire system (as opposed to only the VM and the native compiler). Useful for many applications around certified computations.

The main changes are:

  • a new Int construct to constr and other term representations
  • a new notion of constant implementing a primitive operation (for the type of integers and their operations)
  • specific support in the various reduction machines
  • arithmetic is now 63-bit, instead of 31-bit, and designed to be efficient on 64-bit architectures, and emulated on 32-bit ones (today we do it the other way around)

Still TODO:

  • clean the diff (especially in theories)
  • 32-bit emulation for the VM (native_compute already done)
  • adapt the checker
  • adapt extraction
  • prepare overlays (especially for BigNums)
  • recap the motivations in the commit message
  • fix/check relevant FIXMEs

Depends on #6906 #8773 [merged].

A lot of this code is from @bgregoir.

@maximedenes maximedenes added this to the 8.8+beta1 milestone Mar 5, 2018
@maximedenes maximedenes added the needs: progress Work in progress: awaiting action from the author. label Mar 5, 2018
@Zimmi48 Zimmi48 added kind: feature New user-facing feature request or implementation. needs: rebase Should be rebased on the latest master to solve conflicts or have a newer CI run. needs: overlay This is breaking external developments we track in CI. labels Mar 5, 2018
@Zimmi48
Copy link
Member
Zimmi48 commented Mar 5, 2018

A lot of this code is from @bgregoir.

You could add a commit (to this PR) reflecting this in the dates of his contributions in CREDITS (as is now suggested in the contributing guide).

@gares
Copy link
Member
gares commented Mar 5, 2018

If you want a review in a reasonable time, I think you should split the set of changed files and ask many people (also the competence needed are quite different here and there)

@maximedenes
Copy link
Member Author

you should split the set of changed files

What do you mean by that? There's no way I can make the changes independent in general, so I'm not sure I get the point.

@maximedenes
Copy link
Member Author

What do you mean by that? There's no way I can make the changes independent in general, so I'm not sure I get the point.

Ok, I understand now what you meant. So indeed, maybe @silene can have a look at the VM changes, @ppedrot at primitive contants, and @gares at kernel typing and comparison?

@ejgallego
Copy link
Member

This is likely the most naïve question on earth by why does the kernel needs to be modified?

I would expect that having fast integers at the VM level should suffice for most applications [with a binary representation at the kernel level of course].

@maximedenes
Copy link
Member Author
maximedenes commented Mar 5, 2018

This is likely the most naïve question on earth by why does the kernel needs to be modified?

I would expect that having fast integers at the VM level should suffice for most applications [with a binary representation at the kernel level of course].

I explained that in the various talks I gave on the topic, including during the WG. Maybe you can have a look at the video (although I thought you were attending, but I'm not 100% sure).

@maximedenes
Copy link
Member Author

I would expect that having fast integers at the VM level should suffice for most applications [with a binary representation at the kernel level of course].

The short answer is that it doesn't, in many cases. You can read the corresponding part of my thesis to see citations of work using primitive integers, concretely.

@ejgallego
Copy link
Member

Yeah I know it was explained in the WG talk, but unfortunately I forgot :S

IMHO a small summary of the motivations [in the now extremely terse commit logs] would help preserving this information to future diggers of the codebase. In particular as other systems seems to be pursuing the "fast VM" path.

@maximedenes
Copy link
Member Author

IMHO a small summary of the motivations [in the now extremely terse commit logs] would help preserving this information to future diggers of the codebase. In particular as other systems seems to be pursuing the "fast VM" path.

Ok, will try to improve the messages once the branch stabilizes.

@silene
Copy link
Contributor
silene commented Mar 5, 2018

This is a bit painful to review, since the addition of the Int constructor is mixed with the changes to the VM, hence greatly reducing the signal/noise ratio. Anyway, here are several general remarks or questions (or both) to start with.

  1. GCC has supported uint128_t for eleven years, and Clang is not too far behind. So, the file coq_uint63_native.h should be using this type for architectures that support it rather than handcoding the operations. (The macro is called __SIZEOF_INT128__.)

  2. Since you are changing the opcodes, this is an opportunity to fix what I think was a mistake of the original code. Indeed, arithmetic operations are so cheap that it is faster to execute them twice rather than allocating memory and putting some pressure on the garbage collector. In particular, MULCINT63 should not return a pair but just a single integer representing the high part of the result. Similarly, ADDCINT63 should return a single boolean rather than a variant.
    Hmm...
    Scrap that second remark. The result type of these two opcodes is so hardcoded inside Bignums that this is not worth the effort of fixing it.

  3. What needs LS?INT63CONST1 and why is it important to have two dedicated opcodes?

  4. Instead of prefixing almost all the arithmetic opcodes with CHECK, I would rather have the few unchecked opcodes ADDINT31 and friends be marked as such. Collateral question: what needs these unchecked opcodes? By the way, I am pretty sure that, in practice, the unchecked opcodes will not be any faster than the checked ones, due to branch prediction. (I do hope that no one with a sane mind would use the VM on open arithmetic expressions.)

  5. What the f* is lambda_of_iterator?

@maximedenes
Copy link
Member Author

This is a bit painful to review, since the addition of the Int constructor is mixed with the changes to the VM, hence greatly reducing the signal/noise ratio.

Sorry about that, I tried to do the best I could, initially even the patches adding clambda and changing the representation of values were all mixed... It wasn't easy to reconstruct the history.

I'll try to answer / take into account your remarks after figuring out other problems (I don't know why @bgregoir added the dedicated opcodes you mentioned), but I'm not sure what you mean by:

What the f* is lambda_of_iterator?

lambda_of_iterator is a specialized compilation of iterators on integers. What do you mean exactly?

@silene
Copy link
Contributor
silene commented Mar 5, 2018

lambda_of_iterator is a specialized compilation of iterators on integers. What do you mean exactly?

Why do we need it? Why is it impossible to write it in Coq?

@maximedenes
Copy link
Member Author

Why do we need it? Why is it impossible to write it in Coq?

I think it is for efficiency. If you define it in Coq, you would need to iterate on a type that looks like binary integers I guess, so you'd lose a lot of the benefits. Do you see another way?

@maximedenes
Copy link
Member Author

I looked more into the remaining problems and missing bits, and it seems pretty clear that this is not going to be ready by Thursday, so let's postpone to 8.9. But thanks to all for the constructive feedback! I still hope to make quick progress on this given the maintenance cost.

@maximedenes maximedenes removed this from the 8.8+beta1 milestone Mar 5, 2018
Copy link
Member
@JasonGross JasonGross left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've rebased maximedenes#65 on top of the coqlib changes. Feel free to merge it into this, or to ask me to resubmit it as a PR to Coq after this is merged.

Changes in Ring31.v/Cyclic31. are necessary only if there are custom notations for 0 and 1.
I am baffled by the failure of CompCert, given that it does not show up for me locally, and that maximedenes#67 builds fine for me....

I would prefer to see maximedenes#67 and maximedenes#65 merged before this, but I also don't want to keep getting in the way of this PR being merged.

(((2 ^ (bw - 1) + v) mod (2 ^ bw)) - 2 ^ (bw - 1))%Z.

Definition sto_Z (v : sint) := as_signed 31 (to_Z (unwraps v)).
Unset Printing Records. (* not sure if this is needed; it's here just to be safe *)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not be needed, and I'd rather we test that it is not needed.

@vbgl vbgl force-pushed the primitive-integers branch from f0b47fd to 3be4238 Compare February 1, 2019 10:40
@vbgl
Copy link
Contributor
vbgl commented Feb 1, 2019

I’ve just incorporated the two changes suggested by Jason. CompCert needed both. Thanks.

maximedenes added a commit to maximedenes/HoTT that referenced this pull request Feb 1, 2019
This was a noop and will soon be an error (until `Inductive` properly
supports locality attributes). See rocq-prover/rocq#6914
@vbgl vbgl force-pushed the primitive-integers branch from 3be4238 to 084e9d0 Compare February 1, 2019 14:18
@@ -320,3 +321,6 @@ val entry_has_ident : notation_entry_level -> bool
(** Rem: printing rules for primitive token are canonical *)

val with_notation_protection : ('a -> 'b) -> 'a -> 'b

(** Conversion from bigint to int63 *)
val int63_of_pos_bigint : Bigint.bigint -> Uint63.t
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this belongs in the Uint63 module instead? (Unless we want to keep it outside the kernel.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are no bigints in the kernel.

@ppedrot
Copy link
Member
ppedrot commented Feb 2, 2019

What is the documentation CI failure due to?

@vbgl vbgl force-pushed the primitive-integers branch from 084e9d0 to 807d210 Compare February 2, 2019 17:19
@vbgl
Copy link
Contributor
vbgl commented Feb 4, 2019

@ppedrot No clue. But restarting the pipeline made the problem vanish.

We still have to address Érik’s comment.

vbgl and others added 3 commits February 4, 2019 13:12
This work makes it possible to take advantage of a compact
representation for integers in the entire system, as opposed to only
in some reduction machines. It is useful for heavily computational
applications, where even constructing terms is not possible without such
a representation.

Concretely, it replaces part of the retroknowledge machinery with
a primitive construction for integers in terms, and introduces a kind of
FFI which maps constants to operators (on integers). Properties of these
operators are expressed as explicit axioms, whereas they were hidden in
the retroknowledge-based approach.

This has been presented at the Coq workshop and some Coq Working Groups,
and has been used by various groups for STM trace checking,
computational analysis, etc.

Contributions by Guillaume Bertholon and Pierre Roux <Pierre.Roux@onera.fr>

Co-authored-by: Benjamin Grégoire <Benjamin.Gregoire@inria.fr>
Co-authored-by: Vincent Laporte <Vincent.Laporte@fondation-inria.fr>
@vbgl vbgl force-pushed the primitive-integers branch from c226c52 to 191e253 Compare February 4, 2019 13:13
@vbgl
Copy link
Contributor
vbgl commented Feb 4, 2019

@ppedrot This is now ready.

@ppedrot
Copy link
Member
ppedrot commented Feb 4, 2019

Will merge when the CI agrees.

@maximedenes
Copy link
Member Author

CI agrees!!

@ppedrot ppedrot merged commit 191e253 into rocq-prover:master Feb 4, 2019
ppedrot added a commit that referenced this pull request Feb 4, 2019
Ack-by: JasonGross
Ack-by: SkySkimmer
Ack-by: ejgallego
Ack-by: gares
Ack-by: maximedenes
Ack-by: ppedrot
@maximedenes
Copy link
Member Author

Thanks!

@gares
Copy link
Member
gares commented Feb 4, 2019

Time to do vectors now ;-)

@ckeller
Copy link
Contributor
ckeller commented Feb 5, 2019

Thanks!

@proux01
Copy link
Contributor
proux01 commented Feb 5, 2019

Thanks!

Time to do vectors now ;-)

Would be great indeed.

We are also still working on floats with Érik.

@ejgallego
Copy link
Member

It should be said that many plugins weren't properly ported to this PR as warning 8 is not fatal by default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: feature New user-facing feature request or implementation.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0