[pull] master from JuliaLang:master #2139
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Motivation There are several corner cases in the Julia syntax that are essentially bugs or mistakes that we'd like to possibly remove, but can't due to backwards compatibility concerns. Similarly, when adding new syntax features, there are often cases that overlap with valid (but often nonsensical) existing syntax. In the past, we've mostly done judegement calls of these being "minor changes", but as the package ecosystem grows, so does the chance of someone accidentally using these anyway and our "minor changes" have (subjectively) resulted in more breakages recently. Fortunately, all the recent work on making the parser replacable, combined with the fact that JuliaSyntax already supports parsing multiple revisions of Julia syntax provides a solution here: Just let packages declare what version of the Julia syntax they are using. That way, packages would not break if we make changes to the syntax and they can be upgraded at their own pace the next time the author of that particular package upgrades to a new julia version. # Core mechanism The way this works is simple. Right now, the parser function is always looked up in `Core._parse`. With this PR, it is instead looked up as `mod._internal_julia_parse` (slightly longer name to avoid conflicting with existing bindings of the name in downstream packages), or `Core._parse` if no such binding exists. Similar for `_lower`. There is a macro `@Base.Experimental.set_syntax_version v"1.xx"` that will set the `_internal_julia_parse` (and inte the future the _lower version) to one that propagates the version to the parser, so users are not expected to manipulate the binding directly. # Versioned package loading The loading system is extended to look at a new `syntax.julia_version` key in Project.toml (and Manifest for explicit environments). If no such key exists, it defaults to the minimum allowed version of the Julia compat. If no compat is defined, it defaults to the current Julia version. This is technically slightly less backwards compatible than defaulting this to Julia 1.13, but I think it will be less suprising in the future for the default syntax to match what is in the REPL. Most julia packages do already define a julia compat. Note that as a result of this, the code for parse compat ranges moves from Pkg to Base. # Syntax changes This introduces two parser changes: 1. `@VERSION` (and similar macrocall forms of a macro named `VERSION`) are now special and trigger the parser to push its version information into the source location field of the macrocall. Note that because this is in the parser, this affects all macros with the name. However, there is also logic on the macrocall side that discards this again if the macro cannot accept it. This special mechanism is used by the `Base.Experimental.@VERSION` macro to let users detect the parse version. 2. The `module` syntax form gains a syntax version argument that is automatically populated with the parser's current version. This is the mechanism to propagate syntax information from the parser to the core mechanism above. Note that these are only active if a module has opted into 1.14 syntax, so macros that process `:module` exprs will not see these changes unless and until the calling module opts into 1.14 syntax via the above mentioned mechanisms (which is the primary advantage of this scheme). # Final words I should emphasize that I'm not proposing using this for any big syntax revolutions or anything. I would just like to start cleaning up a few corners of the syntax that I think are universally agreed to be bad but that we've kept for backwards compatibility. This way, by the time we get around to making a breaking revision, our entire ecosystem will have already upgraded to the new syntax.
Closes #59911. `'` is generally treated as its own thing in Expr and flisp, but treated as a special type of call in JuliaSyntax. This change makes `x' = y` parse to the <=1.11 `(= (|'| x) y)` instead of wrapping the body in a block like we do when parsing `f(x) = y`. The special case is fine to me given that it's a more understandable parse, and we're hoping to move away from LineNumberNodes in the future anyway. (I think this was the original reason for the block-wrap for `f(x) = y`).
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )