Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[compiler-v2] Disagreement between ref-safety and bytecode verifier #12701

Closed
wrwg opened this issue Mar 27, 2024 · 0 comments · Fixed by #12757
Closed

[compiler-v2] Disagreement between ref-safety and bytecode verifier #12701

wrwg opened this issue Mar 27, 2024 · 0 comments · Fixed by #12757
Labels
bug Something isn't working compiler-v2

Comments

@wrwg
Copy link
Contributor

wrwg commented Mar 27, 2024

There are various test cases in tests/reference-safety where the checked succeeds bu the bytecode verifier fails. Those need to be investigated and fixed.

@wrwg wrwg added bug Something isn't working compiler-v2 labels Mar 27, 2024
wrwg added a commit that referenced this issue Mar 28, 2024
The freeze operation has some unexpected ad-hoc rules in the v1 borrow semantics: when a reference is frozen, there must not exist any other mutable reference pointing to the same location. It is, however, ok if the currently frozen reference is used later again. This seems to be over-restrictive since it shouldn't matter whether copies of the mutable reference exist as long as they aren't passed at the same time as an argument to a function, i.e. as long as they do not create aliasing.

This PR simulates the v1 borrow rules for freeze, so we avoid bytcode verification errors. At later point we may relax the rules in the bytecode verifier, at which point we could remove those rules or guard them behind a flag.

Closes #12554.  Als repairs some of the bugs in #12701, but not all, so keeping this bug open.
wrwg added a commit that referenced this issue Mar 28, 2024
The freeze operation has some unexpected ad-hoc rules in the v1 borrow semantics: when a reference is frozen, there must not exist any other mutable reference pointing to the same location. It is, however, ok if the currently frozen reference is used later again. This seems to be over-restrictive since it shouldn't matter whether copies of the mutable reference exist as long as they aren't passed at the same time as an argument to a function, i.e. as long as they do not create aliasing.

This PR simulates the v1 borrow rules for freeze, so we avoid bytcode verification errors. At later point we may relax the rules in the bytecode verifier, at which point we could remove those rules or guard them behind a flag.

Closes #12554.  Als repairs some of the bugs in #12701, but not all, so keeping this bug open.
wrwg added a commit that referenced this issue Mar 29, 2024
The freeze operation has some unexpected ad-hoc rules in the v1 borrow semantics: when a reference is frozen, there must not exist any other mutable reference pointing to the same location. It is, however, ok if the currently frozen reference is used later again. This seems to be over-restrictive since it shouldn't matter whether copies of the mutable reference exist as long as they aren't passed at the same time as an argument to a function, i.e. as long as they do not create aliasing.

This PR simulates the v1 borrow rules for freeze, so we avoid bytcode verification errors. At later point we may relax the rules in the bytecode verifier, at which point we could remove those rules or guard them behind a flag.

Closes #12554.  Als repairs some of the bugs in #12701, but not all, so keeping this bug open.
wrwg added a commit that referenced this issue Mar 30, 2024
The freeze operation has some unexpected ad-hoc rules in the v1 borrow semantics: when a reference is frozen, there must not exist any other mutable reference pointing to the same location. It is, however, ok if the currently frozen reference is used later again. This seems to be over-restrictive since it shouldn't matter whether copies of the mutable reference exist as long as they aren't passed at the same time as an argument to a function, i.e. as long as they do not create aliasing.

This PR simulates the v1 borrow rules for freeze, so we avoid bytcode verification errors. At later point we may relax the rules in the bytecode verifier, at which point we could remove those rules or guard them behind a flag.

Closes #12554.  Als repairs some of the bugs in #12701, but not all, so keeping this bug open.
wrwg added a commit that referenced this issue Mar 30, 2024
…on (#12725)

* [compiler-v2] Introduces custom rules for checking the freeze operation

The freeze operation has some unexpected ad-hoc rules in the v1 borrow semantics: when a reference is frozen, there must not exist any other mutable reference pointing to the same location. It is, however, ok if the currently frozen reference is used later again. This seems to be over-restrictive since it shouldn't matter whether copies of the mutable reference exist as long as they aren't passed at the same time as an argument to a function, i.e. as long as they do not create aliasing.

This PR simulates the v1 borrow rules for freeze, so we avoid bytcode verification errors. At later point we may relax the rules in the bytecode verifier, at which point we could remove those rules or guard them behind a flag.

Closes #12554.  Als repairs some of the bugs in #12701, but not all, so keeping this bug open.

* Adding new test case
wrwg added a commit that referenced this issue Apr 1, 2024
…r semantics

This does a number of adaptions to reference safety to deal with some of the unexpected quirks of the v1 borrow semantics. Additional cases are now detected which are logical safe but anyway rejected by v1. Also, the treatment of freeze has been updated to relax some corner cases which led to failures in framework compilation. With this PR, all known bytecode verification failures are now detected except of a few which are sensitive to optimizations on and which pass without optimizations but not without (see #12756).

This closes #12301 and closes #12701

- Fixes `borrow_global` borrow edge to carry a code offset, to be able to distinguish multiple global borrows with different addresses. Since the address in `borrow_global<R>(addr)` is dynamic, the reference checker cannot know whether to borrows results in the same reference. We already tackle this situation for calls which derive reference by adding the code offset, and now do the same for borrow global.
- Makes assignment and let a checkpoint for borrow safety. Until now, we have considered those operations as no-ops for safety (which is safe from memory perspective), but v1 insists on borrow safety for those ops.
- Adds a new check to borrow safety for mut refs which are duplicated and live beyond a safety check point. A particular quirk in v1 is that a mut ref which was used to derive another mut ref is allowed, but one which is duplicated is not. Since the information from which temps refs have been derived is not present in the borrow graph, which is designed to abstract from mut ref copies, we needed to introduce a new field in the borrow state to track this information. Specifically, the following is _not_ allowed in v1: `let r = &mut s; let r1 = r; let x = &mut r.f; *x; *r1`. However, this is allowed: `let r = &mut s; let x = &mut r.f; *x; *r`. The v1 logic behind this seems to be that `x` is derived from `r` but not (for the first example) from `r1`.
wrwg added a commit that referenced this issue Apr 2, 2024
…r semantics

This does a number of adaptions to reference safety to deal with some of the unexpected quirks of the v1 borrow semantics. Additional cases are now detected which are logical safe but anyway rejected by v1. Also, the treatment of freeze has been updated to relax some corner cases which led to failures in framework compilation. With this PR, all known bytecode verification failures are now detected except of a few which are sensitive to optimizations on and which pass without optimizations but not without (see #12756).

This closes #12301 and closes #12701

- Fixes `borrow_global` borrow edge to carry a code offset, to be able to distinguish multiple global borrows with different addresses. Since the address in `borrow_global<R>(addr)` is dynamic, the reference checker cannot know whether to borrows results in the same reference. We already tackle this situation for calls which derive reference by adding the code offset, and now do the same for borrow global.
- Makes assignment and let a checkpoint for borrow safety. Until now, we have considered those operations as no-ops for safety (which is safe from memory perspective), but v1 insists on borrow safety for those ops.
- Adds a new check to borrow safety for mut refs which are duplicated and live beyond a safety check point. A particular quirk in v1 is that a mut ref which was used to derive another mut ref is allowed, but one which is duplicated is not. Since the information from which temps refs have been derived is not present in the borrow graph, which is designed to abstract from mut ref copies, we needed to introduce a new field in the borrow state to track this information. Specifically, the following is _not_ allowed in v1: `let r = &mut s; let r1 = r; let x = &mut r.f; *x; *r1`. However, this is allowed: `let r = &mut s; let x = &mut r.f; *x; *r`. The v1 logic behind this seems to be that `x` is derived from `r` but not (for the first example) from `r1`.
wrwg added a commit that referenced this issue Apr 2, 2024
…r semantics

This does a number of adaptions to reference safety to deal with some of the unexpected quirks of the v1 borrow semantics. Additional cases are now detected which are logical safe but anyway rejected by v1. Also, the treatment of freeze has been updated to relax some corner cases which led to failures in framework compilation. With this PR, all known bytecode verification failures are now detected except of a few which are sensitive to optimizations on and which pass without optimizations but not without (see #12756).

This closes #12301 and closes #12701

- Fixes `borrow_global` borrow edge to carry a code offset, to be able to distinguish multiple global borrows with different addresses. Since the address in `borrow_global<R>(addr)` is dynamic, the reference checker cannot know whether to borrows results in the same reference. We already tackle this situation for calls which derive reference by adding the code offset, and now do the same for borrow global.
- Makes assignment and let a checkpoint for borrow safety. Until now, we have considered those operations as no-ops for safety (which is safe from memory perspective), but v1 insists on borrow safety for those ops.
- Adds a new check to borrow safety for mut refs which are duplicated and live beyond a safety check point. A particular quirk in v1 is that a mut ref which was used to derive another mut ref is allowed, but one which is duplicated is not. Since the information from which temps refs have been derived is not present in the borrow graph, which is designed to abstract from mut ref copies, we needed to introduce a new field in the borrow state to track this information. Specifically, the following is _not_ allowed in v1: `let r = &mut s; let r1 = r; let x = &mut r.f; *x; *r1`. However, this is allowed: `let r = &mut s; let x = &mut r.f; *x; *r`. The v1 logic behind this seems to be that `x` is derived from `r` but not (for the first example) from `r1`.
@wrwg wrwg closed this as completed in b448c40 Apr 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working compiler-v2
Projects
Status: Done
1 participant