Issue #18481 has been updated by jeremyevans0 (Jeremy Evans).


Eregon (Benoit Daloze) wrote in #note-1:
> Would it make sense to automatically download a Rust toolchain when building with YJIT?

I don't think so. Trying to download during a build will result in problems for many packagers.  Ruby doesn't try to download during build for any of the other dependencies it has, and it makes no sense to start doing so now, IMO.

> I assume the Rust code would need a specific version of Rust (newer versions might work but not always, backward compat is not eternal) and so it'd probably need a fixed version to be sure it builds.
> Package managers often only provide a single version of Rust and so would be of little help.

Rust is supposed to be fairly backwards compatible for the last few years.  As long as the YJIT team commits to supporting a Rust version that was released more than 12 months ago (say Rust 1.50), and continues to support that version going forward (as Ruby continues to support C99), I don't expect this to be a significant issue.  However, that should be a hard requirement, we would want to avoid version creep where each new version of Ruby requires a newer version of Rust to build YJIT. If the YJIT team does not think that is feasible, to me that would be a sign that Rust is not yet mature enough for Ruby to rely on.

> In other words, I think it would be nice when building CRuby if it could just download the Rust toolchain it needs, and not require everyone building CRuby to have the right version of Rust, which is likely non-trivial especially for people not familiar at all with Rust.

If Ruby does this, it should only even attempt to do if it is sure in advance that such a toolchain exists for the platform (binary downloads only).  It doesn't makes sense to download and compile Rust itself, considering Rust takes much much longer to build than Ruby.  I still think that is a bad idea, but at least it doesn't hurt platforms that don't have a such a toolchain built for them.

> This would also avoid Ruby installers like ruby-install/ruby-build/RVM/etc to have to add new dependencies, which sounds difficult for e.g. older OS and their package managers which might not have a recent Rust available.
> 
> There could still be a configure option to use an existing Rust toolchain, or automatically detect if the right version is installed.

Or a configure option to opt-in to such toolchain downloading, which IMO would be the best way to support such automated downloading (if we want to support it at all).

> As an alternative, building CRuby (without disabling YJIT) and without the right Rust installed could output instructions for how to install Rust. But this would be much less convenient.

I agree that outputing instructions and stopping the build is a bad idea.  

I think Rust should be treated like OpenSSL. If you build Ruby without OpenSSL, Ruby doesn't download OpenSSL for you, it just doesn't build the openssl extension.  Similarly, if you build Ruby without Rust, Ruby just won't build yjit.  That seems like the simplest and best way to handle things, IMO.  After all, yjit is a purely optional part of Ruby, and use of yjit should be transparent, modulo performance.

----------------------------------------
Feature #18481: Porting YJIT to Rust (request for feedback)
https://bugs.ruby-lang.org/issues/18481#change-95912

* Author: maximecb (Maxime Chevalier-Boisvert)
* Status: Open
* Priority: Normal
----------------------------------------
TL;DR: The YJIT team wants to explore using Rust to help develop YJIT. The rest of CRuby will continue to build without Rust tools and building YJIT will remain optional.

We°«re currently exploring the possibility of porting YJIT to Rust and working on a small proof of concept that should be ready next month. The motivation behind this is that we are facing challenges in terms of code maintainability. As you know, JIT compilers can get very complex, and C99 doesn't offer many tools to manage this complexity. There are no classes and methods, limited type checking, and it's hard to fully separate code into modules, for instance.

We believe that having access to object oriented programming and a more expressive type system would help us manage growing complexity better and also improve the safety/robustness of YJIT. For instance we would like to add Windows support and a new backend to YJIT. That means we°«ll have two separate backends (x86, arm64) and we°«ll need to support two different calling conventions (Microsoft, SystemV), but currently, we have limited tools to build the abstractions needed, such as preprocessor macros and if-statements.

We°«ve discussed the idea of porting YJIT to Rust with some of the Ruby core developers (@ko1, @k0kubun, @mame), and it seems they would be open to merging something like this if it works well. I°«m opening this ticket so that everyone can have a chance to provide feedback and participate in the discussion. We realize that adding Rust code to the CRuby codebase could be challenging and that there are open questions.

We are planning to make it so that YJIT will only need the Rust compiler and `cargo` to build. Building YJIT would then require the Rust compiler to be installed, but CRuby could build without YJIT and without the Rust compiler. There would be no new dependencies for the compiled binary. Rust is supported on Mac, Windows, BSDs, and Linux, which covers all the platforms on which we plan to support YJIT. Since Rust is based on LLVM, it has good support for cross-compilation.

We would like to solicit input from Ruby distributors who create `.deb` and `.rpm` packages. We will likely remain conservative when updating Rust versions to make OS packaging easier. We believe that in the general, the resulting code should be easier to maintain because it will be better organized, but the YJIT team will help out with YJIT-related backports and will be available to help if needed.

Value proposition:
- Rust type systems will catch more bugs early, help prevent new bugs
- Easier to manage growing complexity of YJIT
- Easier to maintain codebase, fewer °»footguns°…
- Easier for newcomers because the compiler catches more bugs
- Better performance because we can implement more sophisticated optimizations
- Easier to add support for new platforms (which adds complexity)
- Rust has mature and easy-to-install tools such as source code formatter and editor plugins
- Rust as a programming language community has a great deal of enthusiasm behind it. This could translate to more enthusiasm for YJIT and for Ruby as well.

Integration:
- YJIT will only depend on the Rust language and the standard library, no other dependencies
- YJIT will be able to build without an internet connection
- Rust has good support for cross-compilation
- Rust is supported on all platforms on which we plan to support with YJIT (Mac, Linux, Windows)
- The compiled CRuby binary won°«t have any new dependencies on shared libraries
- CRuby will still be able to build without `rustc`, with YJIT disabled






-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request / ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>